-- Logs begin at Wed 2022-07-20 21:08:30 UTC, end at Wed 2022-07-20 22:18:21 UTC. -- Jul 20 21:38:07.357966 dv-cinder-849690-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 21:38:07.365497 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 21:38:07 2022] *** Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: nodename: dv-cinder-849690-5-iscsi Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: machine: x86_64 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: clock source: unix Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: pcre jit disabled Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: detected number of CPU cores: 4 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: current working directory: / Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: detected binary path: /usr/bin/uwsgi-core Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: your processes number limit is 47680 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: your memory page size is 4096 bytes Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: detected max file descriptor number: 2048 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: lock engine: pthread robust mutexes Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: thunder lock: enabled Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 21:38:07.369189 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 21:38:07.412803 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: Python main interpreter initialized at 0x55d797d3d9c0 Jul 20 21:38:07.412803 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: python threads support enabled Jul 20 21:38:07.412803 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: your server socket listen backlog is limited to 100 connections Jul 20 21:38:07.412803 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: your mercy for graceful operations on workers is 90 seconds Jul 20 21:38:07.413131 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 21:38:07.413236 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: *** Operational MODE: preforking *** Jul 20 21:38:07.413507 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: *** uWSGI is running in multiple interpreter mode *** Jul 20 21:38:07.413507 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: spawned uWSGI master process (pid: 86623) Jul 20 21:38:07.413705 dv-cinder-849690-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 21:38:07.419083 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: spawned uWSGI worker 1 (pid: 86635, cores: 1) Jul 20 21:38:07.419083 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: spawned uWSGI worker 2 (pid: 86636, cores: 1) Jul 20 21:38:07.419083 dv-cinder-849690-5-iscsi devstack@keystone.service[86623]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 21:38:12.823321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 21:38:12.823321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: from cryptography.utils import int_from_bytes Jul 20 21:38:12.824119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 21:38:12.824119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: from cryptography.utils import int_from_bytes Jul 20 21:38:12.825756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 21:38:12.825756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: from cryptography.utils import int_from_bytes Jul 20 21:38:12.826210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 21:38:12.826210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: from cryptography.utils import int_from_bytes Jul 20 21:38:12.875702 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 21:38:12.875906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 21:38:12.876085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] command line args: [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 21:38:12.876354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 21:38:12.876535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ================================================================================ {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 21:38:12.876791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] admin_token = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.877037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] config_dir = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.877221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.877402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] config_source = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.877578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] debug = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.877775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.878957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.878957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.878957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.878957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.879248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.879248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_config_append = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.879365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.879522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_dir = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.879697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_file = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.879932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.880180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.880426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.880671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.880846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.881021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.881262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.881510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.881757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.881990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.882222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.882452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.882697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.882802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 21:38:12.882942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.883022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 21:38:12.883208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] command line args: [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 21:38:12.883289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.883479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 21:38:12.883656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ================================================================================ {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 21:38:12.883747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.883868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] publish_errors = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.883943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] admin_token = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] config_dir = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] config_source = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] debug = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.884898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] standard_threads = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.885977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] use_journal = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] use_json = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_config_append = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] use_stderr = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_dir = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] use_syslog = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_file = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.886926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.887098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.887182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.887345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.887434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.887585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.887678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.887758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.887879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.887979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.888120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.888210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.888369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.888448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.888598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.888684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.888826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.888971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.889055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.889129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.889268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.889347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.889506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.889599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.889746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.889833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.889981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.890059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.890228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.890474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.890559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.86/identity {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.890696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] publish_errors = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.890782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.890941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.891020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.891282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.891472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.891558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.891660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.891748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.891911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.891997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.892147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.892226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.892378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] standard_threads = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.892465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.892611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.892690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.892852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.892931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.893011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.893140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.893219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] use_journal = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.893343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] use_json = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.893421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.893501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] use_stderr = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.893582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.893660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] use_syslog = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.893824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.893910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 21:38:12.894072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.894994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.896888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.896984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.897957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898263 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.898938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.899855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.900949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.901896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.902987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.903930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.904961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.905881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.906958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.907994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.908901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.909984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.910920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.external = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.911964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.password = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] auth.token = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.912882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.913966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.914968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.915911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.916580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.916694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.916866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.916997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.917959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.external = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.918913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.password = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.919969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] auth.token = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.920901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921152 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.921997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.922945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.923928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.924957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.925976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.926934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927424 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.927904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.928857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.929952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.930887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.931014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.931179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.931259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.931377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.931479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.931615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.932405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.932507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.932671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.932753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.932958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.933925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.934939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.935921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.936863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.937933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.938952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.939909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.940852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.941933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.942970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.943920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.944858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] role.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.945966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] role.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.946964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.947948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.948961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.949853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.950978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.951956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] role.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] role.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.952965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.953912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.954851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.955966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.956902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.957875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.connection = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.958982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.959958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.960966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.961901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.962956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86636) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 21:38:12.963099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.963282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.963515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.963748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.963979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.964220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.964463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.964690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.964927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.965219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.965475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.connection = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.965715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.965959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.966191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.966426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.966655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.966886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.967140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.967373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.967613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.967844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.968084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.968313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.968556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.968787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.969024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.969249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.969483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.969711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 21:38:12.969875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 21:38:12.994074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:12.994866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:12.995130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:12.997069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:12.997295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:12.997517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.003479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.005813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.031924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.032153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.033623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.033851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.178699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.179405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.227454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.227683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.228211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.228447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.293966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.294092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.294187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.294306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.294521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.318855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.319953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.320039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.320162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.320248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.320372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.331214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.331630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.344491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.344717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.344860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.344946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.345937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.346151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86636) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 21:38:13.363528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.363737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 21:38:13.363827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.364008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 21:38:13.364118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.364242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 21:38:13.364327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.364508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.364593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 21:38:13.364766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.364860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.365021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.365107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.365272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.365356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.365524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.376326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.376601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.376687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.376867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.376953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.377133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.377217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.377417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.385244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.385579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.388532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.388810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.388894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.389140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.389223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.389417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.389502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.389724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.389807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.389982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.390885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.391079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.391167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.391352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.391436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.391616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.418782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.419550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.420429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.420703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.421159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.421443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.425940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.426214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.426490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.426699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.426976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.427281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.433741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.434019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.434852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.435161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.438608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.438889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.439192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.439473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.439751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.439835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.440023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.440108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.440287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.440371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.440545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.440631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.440839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.441111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.441379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.441643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.459396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.459681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.460747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.461023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.464926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.466189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.468540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.468833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.469843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.470140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.473817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.475175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.477451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.479370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.481890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.482177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.482463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.482745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.483025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.483136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.483324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.483411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.483697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.484242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.484527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.484814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.502515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.502887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.503194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.503456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.503844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.504219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.504490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.504755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.511861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.513499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.513587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.513802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.515228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.515505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.516869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.517146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.517422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.517697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.517966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.518604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.518884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.519186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.519585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.519867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.531437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.531714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.531998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.532269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.532543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.532829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.533099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.533317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.533595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.533883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.534162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.534440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.534728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.535002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.552731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.554811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.556013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.558770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.559959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.561557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.562112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.563203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.563480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.563741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.563826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.565372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.565650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.565937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.571289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.573131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.573494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.575133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.576354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.576637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.576911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.577186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.578335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.578625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.578913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.579214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.586419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.586659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.586933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.588490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.588773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.589050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.594789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.595086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.595362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.595649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.595898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.596170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.596446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.596713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.596928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86636) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 21:38:13.597014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.597204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.597289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.597479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.597570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.597734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.597819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.598002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.598140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.598272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.598357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.598554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.598822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.599113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.599385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.599657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.599932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.600195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.600462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86636) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 21:38:13.638385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d797d3d9c0 pid: 86635 (default app) Jul 20 21:38:13.640521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d797d3d9c0 pid: 86636 (default app) Jul 20 21:38:13.641693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67953fb-43cc-4cf7-a865-bd4d4a0c0a9f None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:13.641983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67953fb-43cc-4cf7-a865-bd4d4a0c0a9f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:13.642262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67953fb-43cc-4cf7-a865-bd4d4a0c0a9f None None] PATH_INFO: `/v3` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:13.646465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1/1] 10.222.0.86 () {50 vars in 791 bytes} [Wed Jul 20 21:38:13 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:21.311856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:21.312409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:21.312409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:21.316745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1/2] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:21.322737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:21.323200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:21.323315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:21.392547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: INFO dbcounter [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] Registered counter for database keystone Jul 20 21:38:21.413212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG oslo_db.sqlalchemy.engines [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86635) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 21:38:21.419543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writer thread running {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 21:38:21.594656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG passlib.handlers.bcrypt [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86635) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 21:38:21.595093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG passlib.handlers.bcrypt [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86635) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 21:38:21.649621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:21.651244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:21.777956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 2/3] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 20 21:38:21.942834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:22.013116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: INFO dbcounter [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] Registered counter for database keystone Jul 20 21:38:22.031850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG oslo_db.sqlalchemy.engines [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86636) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 21:38:22.037880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writer thread running {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 21:38:22.156926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d81a1-84b0-40c5-a313-52baba8dd195 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:22.161155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:22.161814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:22.162097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:22.162373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:22.171240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:22.222615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 21:38:22.222951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998aeee-aa57-4fd4-8dda-ea3b89d92198 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:22.233205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 2/4] 10.222.0.86 () {58 vars in 1324 bytes} [Wed Jul 20 21:38:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 21:38:22.239524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:22.254202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3bbb3-f582-44a5-8e03-66ce6fb3d1a5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:22.258646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:22.260067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:22.260380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:22.260661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] PATH_INFO: `/v3/projects/feb77f81bd3a4077b594b0faf70919df` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:22.264747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] RBAC: Authorizing `identity:get_project(project_id=feb77f81bd3a4077b594b0faf70919df)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:22.315723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING keystone.common.rbac_enforcer.enforcer [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 21:38:22.316070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375eeaf3-f44b-4861-b5c4-c7d358808346 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:22.320695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 3/5] 10.222.0.86 () {56 vars in 1207 bytes} [Wed Jul 20 21:38:22 2022] GET /identity/v3/projects/feb77f81bd3a4077b594b0faf70919df => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:24.344891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d663b-c4b7-47d4-a1c4-7b6ab5a87251 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:24.345392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d663b-c4b7-47d4-a1c4-7b6ab5a87251 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:24.345392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d663b-c4b7-47d4-a1c4-7b6ab5a87251 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:24.349554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 3/6] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:24.355672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:24.356071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:24.356233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:24.374966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:24.376586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:24.384768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 4/7] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 21:38:24.407325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d663b-c4b7-47d4-a1c4-7b6ab5a87251 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:24.420069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d663b-c4b7-47d4-a1c4-7b6ab5a87251 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:24.424309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e031fc8-a5b8-4bb7-8d03-3a47a2a73186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:24.424942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e031fc8-a5b8-4bb7-8d03-3a47a2a73186 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:24.425228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e031fc8-a5b8-4bb7-8d03-3a47a2a73186 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:24.425509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e031fc8-a5b8-4bb7-8d03-3a47a2a73186 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:24.433328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e031fc8-a5b8-4bb7-8d03-3a47a2a73186 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:24.435137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e031fc8-a5b8-4bb7-8d03-3a47a2a73186 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:24.443855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 4/8] 10.222.0.86 () {58 vars in 1324 bytes} [Wed Jul 20 21:38:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:38:24.449404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:24.458906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbf016-aee6-4294-8f89-c49fa00763bc None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:24.464027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e717c11-e96d-4f80-b0ec-e392f95488fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:24.464892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717c11-e96d-4f80-b0ec-e392f95488fd admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:24.465179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717c11-e96d-4f80-b0ec-e392f95488fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:24.465455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e717c11-e96d-4f80-b0ec-e392f95488fd admin admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:24.466860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e717c11-e96d-4f80-b0ec-e392f95488fd admin admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:24.468753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e717c11-e96d-4f80-b0ec-e392f95488fd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:24.470327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 5/9] 10.222.0.86 () {56 vars in 1198 bytes} [Wed Jul 20 21:38:24 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:26.528242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ef81-4fa2-4ff8-842e-56fd0c87c14a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.528772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ef81-4fa2-4ff8-842e-56fd0c87c14a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.528772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ef81-4fa2-4ff8-842e-56fd0c87c14a None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.530147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 5/10] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:26.535332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.535751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.536005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.552801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:26.553972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:26.596946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7e458-174c-493d-8cab-46e3ea826a87 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.597234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7e458-174c-493d-8cab-46e3ea826a87 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.597517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7e458-174c-493d-8cab-46e3ea826a87 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.598876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 6/11] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:26.604534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.604900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.605187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.635194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 6/12] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:26.645453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG passlib.handlers.bcrypt [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86636) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 21:38:26.645872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG passlib.handlers.bcrypt [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86636) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 21:38:26.658164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:26.682104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:26.692124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:26.693754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:26.764105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bdbae0-0d90-4555-a248-9ab37be0bd74 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:26.795718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 7/13] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:26.802511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:26.803418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.803771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.804119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.816053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:26.816991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:26.827873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:26.828991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:26.829904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 7/14] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:38:26.836434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:26.918460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc41ae-9022-45fc-8934-43aaa52ad2b5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:26.920367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:26.920769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:26.939859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce05fa-a20b-4c5f-8ab2-694f642ca313 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:26.956454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:26.957348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.957633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.957909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.973640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:26.974301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:26.974582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:26.974857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:26.975637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:26.975944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:26.977346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:26.989621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 8/15] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:26.995295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:27.031170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:27.032710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:27.096977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47487e0f-3b4f-4aeb-96f3-382fd94cc573 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:27.130477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:27.131390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:27.131677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:27.131840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 8/16] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:38:27.135151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:27.136446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:27.137921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:27.138694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:27.139593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 9/17] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:26 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:27.142078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:27.144668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:27.159420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:27.238006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f8cef-c0ad-4e6c-9801-81e0d6620812 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:27.246208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51948f27-a26a-4ddb-9ff3-222a1137e476 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:27.277864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:27.278659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:27.278939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:27.279240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:27.280265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:27.281052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:27.281332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:27.281600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:27.282772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:27.284261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:27.285848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 10/18] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:27.292009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:27.299462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:27.300941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:38:27.321733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:27.322656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:27.323522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:38:27.323522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:38:27.323522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:38:27.323522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:27.323522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 9/19] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:38:27.329016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:27.374064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:27.384575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:27.393570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1368d6-1d0f-4f4a-a3dc-afca7ee2a3c8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:27.427386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:27.428042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:27.428342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:27.428615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9409498-fddd-4bfb-a18c-21c5e38bda28 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:27.428710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:27.429439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.domain.id=default, user.id=ec67f4a1435249d49af5a7f7df4f7a39) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:27.429747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:27.431236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2562ed6-8426-4743-943b-c9f50145f9cf None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:27.437963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 11/20] 10.222.0.86 () {56 vars in 1322 bytes} [Wed Jul 20 21:38:27 2022] GET /identity/v3/role_assignments?user.id=ec67f4a1435249d49af5a7f7df4f7a39&scope.domain.id=default&role.id=62c943ce95d54a1999a447dca6a1103b => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:27.469020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:27.469704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:27.469987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:27.470262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:27.471007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:27.471345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:27.472779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190a237-ff3f-445a-b7bd-31ba0558d42c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:27.493674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 10/21] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:29.670559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73e08-4b42-452c-b9e5-f2f585efbcc4 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:29.671596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73e08-4b42-452c-b9e5-f2f585efbcc4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:29.672096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c73e08-4b42-452c-b9e5-f2f585efbcc4 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:29.673685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 12/22] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:29.679280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:29.679683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:29.679911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:29.708811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:29.710036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:29.804474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 11/23] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:29.852514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c73e08-4b42-452c-b9e5-f2f585efbcc4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:30.001273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c73e08-4b42-452c-b9e5-f2f585efbcc4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:30.047554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7493-3b67-498a-be9d-890c94bcacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:30.048720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7493-3b67-498a-be9d-890c94bcacc7 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.049193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7493-3b67-498a-be9d-890c94bcacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.049663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7493-3b67-498a-be9d-890c94bcacc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.071981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7493-3b67-498a-be9d-890c94bcacc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-86bc7493-3b67-498a-be9d-890c94bcacc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:30.083088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:30.083932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:30.084716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 13/24] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:30.088719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:30.157480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:30.160258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de949e47-b2ec-42bf-b87f-6dd7903970fa None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.160804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de949e47-b2ec-42bf-b87f-6dd7903970fa None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.161274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de949e47-b2ec-42bf-b87f-6dd7903970fa None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.163601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 14/25] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:30.170393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10e2d5-bbe0-42a5-b66d-6ce7e7b761b3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.170942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10e2d5-bbe0-42a5-b66d-6ce7e7b761b3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.171428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10e2d5-bbe0-42a5-b66d-6ce7e7b761b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.199447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:30.201592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7e10e2d5-bbe0-42a5-b66d-6ce7e7b761b3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:30.202814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7e10e2d5-bbe0-42a5-b66d-6ce7e7b761b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:30.268348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96b4cb-e5d8-4e83-b293-70955e61973f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:30.338776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:30.339493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.339786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.340072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.340840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:30.341153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:30.342560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:30.345860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 15/26] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:30.347220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3c367-658f-464f-9614-ed65b0323e77 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.347509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3c367-658f-464f-9614-ed65b0323e77 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.347785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3c367-658f-464f-9614-ed65b0323e77 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.349062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 16/27] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:30.355701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.355990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.356270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.376973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 12/28] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:30.384026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:30.388628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:30.389907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:30.536471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 17/29] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:30.537259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:30.592506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3225c-7d2e-4052-8313-9e30c2a16f99 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:30.602850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:30.637675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:30.638984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:30.640535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.641002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.641434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.656502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:30.658736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:30.684065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5706a8-789c-4ece-90e8-bd3acd1eabe4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:38:30.687573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:30.688669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:30.689569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:38:30.689569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:38:30.689569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:38:30.689569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:30.691606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 13/30] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:30.692605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:30.727322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2bc15-33bc-4217-aa4d-783efa346278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:30.729278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2bc15-33bc-4217-aa4d-783efa346278 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.729562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2bc15-33bc-4217-aa4d-783efa346278 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.729837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2bc15-33bc-4217-aa4d-783efa346278 None admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.731265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2bc15-33bc-4217-aa4d-783efa346278 None admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:30.733462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2bc15-33bc-4217-aa4d-783efa346278 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:30.739123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 18/31] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:30 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:30.740634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f598cb6-fa31-4caf-b5a9-8095f8f3979e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.743202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f598cb6-fa31-4caf-b5a9-8095f8f3979e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.743490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f598cb6-fa31-4caf-b5a9-8095f8f3979e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.744973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 19/32] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:30.747748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f598cb6-fa31-4caf-b5a9-8095f8f3979e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:30.835499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfcdde-8027-4f29-8a18-978c6f1fc1da None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:30.881205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79715337-7091-41fd-8ea5-ef0523530273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:30.881906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79715337-7091-41fd-8ea5-ef0523530273 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.882198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79715337-7091-41fd-8ea5-ef0523530273 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.882476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79715337-7091-41fd-8ea5-ef0523530273 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.883165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f598cb6-fa31-4caf-b5a9-8095f8f3979e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:30.883272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79715337-7091-41fd-8ea5-ef0523530273 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:30.885194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79715337-7091-41fd-8ea5-ef0523530273 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:30.920539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:30.927453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:30.928463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:30.928979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:30.929529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:30.930492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:30.931022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:30.933529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:30.954658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 20/33] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:30.956025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:31.032407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 14/34] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Jul 20 21:38:30 2022] POST /identity/v3/roles => generated 225 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:31.034280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.034793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.035353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.066223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:31.070023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:31.113943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:31.120324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed6acf-7c60-401b-b6e2-ab448fa1d59f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:31.172797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:31.173615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.173898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.174170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.179918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:31.181964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:31.188446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 21/35] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:31.188446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:31.219226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 15/36] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:31.220012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:31.292426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:31.352929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35b3c3-a714-4e80-bd20-efb9b4fd275d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:31.362824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d64496-4bcf-4f58-8652-3265e709d869 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:31.391753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f18dd-7083-4cdc-ba32-78ebbdb1efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:31.392340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f18dd-7083-4cdc-ba32-78ebbdb1efc5 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.392621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f18dd-7083-4cdc-ba32-78ebbdb1efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.392899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f18dd-7083-4cdc-ba32-78ebbdb1efc5 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.393641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f18dd-7083-4cdc-ba32-78ebbdb1efc5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:31.395127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f18dd-7083-4cdc-ba32-78ebbdb1efc5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:31.400276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ea9de-7c02-4b14-8374-845055e82421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:31.401158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ea9de-7c02-4b14-8374-845055e82421 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.401439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ea9de-7c02-4b14-8374-845055e82421 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.401720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ea9de-7c02-4b14-8374-845055e82421 None admin] PATH_INFO: `/v3/domains/default/users/ec67f4a1435249d49af5a7f7df4f7a39/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.404171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ea9de-7c02-4b14-8374-845055e82421 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ec67f4a1435249d49af5a7f7df4f7a39, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:31.405769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ea9de-7c02-4b14-8374-845055e82421 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:31.423024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 16/37] 10.222.0.86 () {58 vars in 1361 bytes} [Wed Jul 20 21:38:31 2022] PUT /identity/v3/domains/default/users/ec67f4a1435249d49af5a7f7df4f7a39/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:38:31.424730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796ed1-a989-431f-80be-a742a4fbfe2d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.425307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796ed1-a989-431f-80be-a742a4fbfe2d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.425738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796ed1-a989-431f-80be-a742a4fbfe2d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.427643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 17/38] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:31.428574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796ed1-a989-431f-80be-a742a4fbfe2d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:31.448911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:31.534398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 22/39] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:38:31 2022] POST /identity/v3/projects => generated 301 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:31.536152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.536517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.536870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.561863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:31.563676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:31.569493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796ed1-a989-431f-80be-a742a4fbfe2d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:31.628905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d593c7-ffcf-4f07-8852-9f3f38b3e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:31.629947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d593c7-ffcf-4f07-8852-9f3f38b3e405 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.630446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d593c7-ffcf-4f07-8852-9f3f38b3e405 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.630905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d593c7-ffcf-4f07-8852-9f3f38b3e405 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.631937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d593c7-ffcf-4f07-8852-9f3f38b3e405 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:31.633405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d593c7-ffcf-4f07-8852-9f3f38b3e405 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:31.652012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 18/40] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Jul 20 21:38:31 2022] POST /identity/v3/roles => generated 219 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:31.653379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707cb98-e464-495d-82d7-f2b9b5978a69 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.653669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707cb98-e464-495d-82d7-f2b9b5978a69 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.653953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707cb98-e464-495d-82d7-f2b9b5978a69 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.655416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 19/41] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:31.662056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.662509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.662865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.686190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:31.687720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:31.786838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 23/42] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:31.804051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:31.810303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 20/43] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:31.824672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:31.839955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:31.885557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:31.957110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ac4bf-a862-423c-a887-ebf32b988bcc None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:31.971837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673be110-7623-46dc-a4d1-c8d76df4d540 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:31.996092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:31.996902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:31.997191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:31.997472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:31.998669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:32.000565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:32.002148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 24/44] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:32.007432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:32.012195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a17c90-bf45-42f6-a08f-10a70e433bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:32.012872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a17c90-bf45-42f6-a08f-10a70e433bae None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:32.013189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a17c90-bf45-42f6-a08f-10a70e433bae None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:32.013435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a17c90-bf45-42f6-a08f-10a70e433bae None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:32.014207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a17c90-bf45-42f6-a08f-10a70e433bae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:32.015639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a17c90-bf45-42f6-a08f-10a70e433bae None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:32.032870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 21/45] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Jul 20 21:38:31 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:32.172240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782b80a-9c6a-480e-bcec-74db1c138298 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:32.184284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:32.225161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6270a5-2adc-489a-b3dd-984dc0580690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:32.225884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6270a5-2adc-489a-b3dd-984dc0580690 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:32.226177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6270a5-2adc-489a-b3dd-984dc0580690 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:32.226460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6270a5-2adc-489a-b3dd-984dc0580690 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:32.227576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6270a5-2adc-489a-b3dd-984dc0580690 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:32.229416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6270a5-2adc-489a-b3dd-984dc0580690 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:32.263337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 25/46] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:38:32 2022] POST /identity/v3/projects => generated 312 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:38:33.623512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80dc3f-2478-4254-803e-f6014b9fa74d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:33.624030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80dc3f-2478-4254-803e-f6014b9fa74d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:33.624111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80dc3f-2478-4254-803e-f6014b9fa74d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:33.625459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 22/47] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:33.630553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af3e0-4185-49a6-a266-26807378d017 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:33.630974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af3e0-4185-49a6-a266-26807378d017 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:33.631206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606af3e0-4185-49a6-a266-26807378d017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:33.648135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-606af3e0-4185-49a6-a266-26807378d017 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:33.649306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-606af3e0-4185-49a6-a266-26807378d017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:33.713911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 26/48] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:33.737278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80dc3f-2478-4254-803e-f6014b9fa74d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:33.798383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:33.840466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80dc3f-2478-4254-803e-f6014b9fa74d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:33.874479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:33.875235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:33.875521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:33.875793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:33.887949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:33.896716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:33.897576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:33.898415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 23/49] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:33.904046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af3e0-4185-49a6-a266-26807378d017 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:33.993199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:34.003355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606af3e0-4185-49a6-a266-26807378d017 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:34.035698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:34.036367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.036648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.036925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.037680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:34.037985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:34.039406 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:34.048294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 27/50] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:34.053825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:34.154390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5ea95-2c9b-4686-bee2-aabd0019734f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:34.173739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:34.187961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:34.188823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.189099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.189373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.190573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:34.192066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:34.193694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 24/51] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:34 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:34.199178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:34.296973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2347cfb-9b0f-4201-8cb8-4a9e914c31c1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:34.328541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a1bd3-0fa2-4202-9831-726d3e7cd928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:34.329333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a1bd3-0fa2-4202-9831-726d3e7cd928 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.329616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a1bd3-0fa2-4202-9831-726d3e7cd928 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.329891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a1bd3-0fa2-4202-9831-726d3e7cd928 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.331068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a1bd3-0fa2-4202-9831-726d3e7cd928 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:34.332549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a1bd3-0fa2-4202-9831-726d3e7cd928 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:34.334165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 28/52] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:34.340428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:34.368461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b15474-e10f-468e-b376-20d11fb43d22 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.369056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b15474-e10f-468e-b376-20d11fb43d22 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.369344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b15474-e10f-468e-b376-20d11fb43d22 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.370699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 29/53] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:34.375592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.375880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.376159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.392981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:34.394176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:34.411130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:34.443003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c911c5-d239-43bf-98f2-ada16638ba01 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:34.466316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 30/54] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:34.476418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:34.477091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.477380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.477664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.479432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.domain.id=default, user.id=ec67f4a1435249d49af5a7f7df4f7a39) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:34.479760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:34.481229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:34.503089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 25/55] 10.222.0.86 () {56 vars in 1322 bytes} [Wed Jul 20 21:38:34 2022] GET /identity/v3/role_assignments?user.id=ec67f4a1435249d49af5a7f7df4f7a39&scope.domain.id=default&role.id=62c943ce95d54a1999a447dca6a1103b => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:34.503296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:34.595336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc651b88-7604-43c3-955c-9a617d5bff6e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:34.630741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5496e-7b4a-47e7-83dd-e9a1a3add143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:34.631587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5496e-7b4a-47e7-83dd-e9a1a3add143 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.631874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5496e-7b4a-47e7-83dd-e9a1a3add143 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.632155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5496e-7b4a-47e7-83dd-e9a1a3add143 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.633360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5496e-7b4a-47e7-83dd-e9a1a3add143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:34.635366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5496e-7b4a-47e7-83dd-e9a1a3add143 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:34.637014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 31/56] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:34.642603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:34.801786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:34.848078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1d6de-9449-4a08-8d6a-276d71f67fc3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:34.880556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd0f19-d625-4273-9720-27e773590d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:34.881210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd0f19-d625-4273-9720-27e773590d64 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:34.881495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd0f19-d625-4273-9720-27e773590d64 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:34.881775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd0f19-d625-4273-9720-27e773590d64 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:34.882496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd0f19-d625-4273-9720-27e773590d64 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:34.883956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bd0f19-d625-4273-9720-27e773590d64 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:34.910400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 26/57] 10.222.0.86 () {60 vars in 1140 bytes} [Wed Jul 20 21:38:34 2022] POST /identity/v3/projects => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:36.924709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b70e8-c5c6-47b1-a068-f25d8f8d944a None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:36.925224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b70e8-c5c6-47b1-a068-f25d8f8d944a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:36.925307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8b70e8-c5c6-47b1-a068-f25d8f8d944a None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:36.926652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 32/58] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:36.931837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:36.932233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:36.932397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:36.949612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:36.950872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:37.016280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 27/59] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:37.040128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b70e8-c5c6-47b1-a068-f25d8f8d944a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:37.062390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:37.143419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8b70e8-c5c6-47b1-a068-f25d8f8d944a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:37.176992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31a143-0f75-4292-926d-c05bd5b7b0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:37.181526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31a143-0f75-4292-926d-c05bd5b7b0d5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:37.181526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31a143-0f75-4292-926d-c05bd5b7b0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:37.181526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31a143-0f75-4292-926d-c05bd5b7b0d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:37.181526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31a143-0f75-4292-926d-c05bd5b7b0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:37.181526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31a143-0f75-4292-926d-c05bd5b7b0d5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:37.182498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 33/60] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:37.188047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:37.220993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:37.286867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d074a-554b-4c37-9eeb-3b1b81e8c2fa None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:37.325003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7326308-43a1-4a2e-b8cf-fdf919487709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:37.326123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326308-43a1-4a2e-b8cf-fdf919487709 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:37.326534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326308-43a1-4a2e-b8cf-fdf919487709 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:37.326820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7326308-43a1-4a2e-b8cf-fdf919487709 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:37.327848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7326308-43a1-4a2e-b8cf-fdf919487709 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:37.329318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7326308-43a1-4a2e-b8cf-fdf919487709 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:37.358948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 28/61] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:38:37 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:40.409670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e122521-e882-400d-ad0f-d006f251163e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.410648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e122521-e882-400d-ad0f-d006f251163e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.411155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e122521-e882-400d-ad0f-d006f251163e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.412729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 34/62] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:40.418225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.419039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.419550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.444040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:40.445552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:40.537345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 29/63] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:40.574418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e122521-e882-400d-ad0f-d006f251163e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:40.695803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:40.714408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e122521-e882-400d-ad0f-d006f251163e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:40.764479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57c65f-39b9-4a73-a4af-6f3d56250261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:40.765364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57c65f-39b9-4a73-a4af-6f3d56250261 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.765650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57c65f-39b9-4a73-a4af-6f3d56250261 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.765928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57c65f-39b9-4a73-a4af-6f3d56250261 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.781851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57c65f-39b9-4a73-a4af-6f3d56250261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-8e57c65f-39b9-4a73-a4af-6f3d56250261 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:40.790106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:40.791532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:40.792805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 35/64] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:38:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:40.800402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:40.807910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ca6df-06b6-47cb-aa1a-919c797204b8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.808200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ca6df-06b6-47cb-aa1a-919c797204b8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.808482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ca6df-06b6-47cb-aa1a-919c797204b8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.809895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 36/65] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:40.820238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020540dd-02a6-4d49-8e49-d463fcedc1db None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.820531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020540dd-02a6-4d49-8e49-d463fcedc1db None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.820816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020540dd-02a6-4d49-8e49-d463fcedc1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.838108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:40.841156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-020540dd-02a6-4d49-8e49-d463fcedc1db None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:40.842584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-020540dd-02a6-4d49-8e49-d463fcedc1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:40.927417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 37/66] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:40.935363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af26f-25e4-4e8a-9784-8abd22e33e31 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.935937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af26f-25e4-4e8a-9784-8abd22e33e31 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.936430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040af26f-25e4-4e8a-9784-8abd22e33e31 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.937989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 38/67] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:40.943034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:40.943611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:40.944091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:40.962180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:40.963748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:41.000143 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185371f-e1d4-4be5-948d-c1bc3b49eb56 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.038363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 39/68] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:41.038538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.039357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.039806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.039916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.040398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.041377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:41.041828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:41.043619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917eeab5-da34-4499-aa75-6c01963db564 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:41.052665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 30/69] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:38:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:41.057869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.059496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93016c6e-655d-421b-8570-ac4c694089ed None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.059962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93016c6e-655d-421b-8570-ac4c694089ed None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.060383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93016c6e-655d-421b-8570-ac4c694089ed None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.061951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 31/70] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:41.062841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93016c6e-655d-421b-8570-ac4c694089ed None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.152624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.160071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b63e3-b2dd-4e0f-a3ef-75fa65bb1b9d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.176018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93016c6e-655d-421b-8570-ac4c694089ed None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.194093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233fa68d-434c-4559-a77a-7d60bf7b3e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.194886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa68d-434c-4559-a77a-7d60bf7b3e4a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.195206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa68d-434c-4559-a77a-7d60bf7b3e4a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.195493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233fa68d-434c-4559-a77a-7d60bf7b3e4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.206035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233fa68d-434c-4559-a77a-7d60bf7b3e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:41.209354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.210225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.210510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.210786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-233fa68d-434c-4559-a77a-7d60bf7b3e4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:41.214244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:41.215137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:41.215922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 40/71] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:41.216577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31827488-c584-4173-bb65-400563cb8105 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.221786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31827488-c584-4173-bb65-400563cb8105 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.221786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31827488-c584-4173-bb65-400563cb8105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.225559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:41.227029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:41.229489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 32/72] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:41.230254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.237625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-31827488-c584-4173-bb65-400563cb8105 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:41.238821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-31827488-c584-4173-bb65-400563cb8105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:41.299952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.305795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 41/73] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:41.307725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fafd57-26d7-4362-9a90-dc4b17b4948f None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.308012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fafd57-26d7-4362-9a90-dc4b17b4948f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.308285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fafd57-26d7-4362-9a90-dc4b17b4948f None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.309559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 42/74] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:41.310249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fafd57-26d7-4362-9a90-dc4b17b4948f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.338031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c75887-56aa-4d2a-b856-71f20121b510 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.374484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.375312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.375604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.375883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.384144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.387712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:41.395609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:41.396642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 33/75] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:41.399773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.417876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fafd57-26d7-4362-9a90-dc4b17b4948f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.451763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.452438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.452729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.453018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.453770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:41.454082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:41.455518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db7d65-1412-401a-acb7-d178e9929164 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:41.463618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 43/76] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:41.465106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.465338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.465618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.481543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:41.482715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:41.499332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07781bc8-04e2-4b18-9fde-566f821e295b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.531847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.532494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.532776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.533051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.538608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.552185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 44/77] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:41.553003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.601268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.604446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:41.607178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:41.652474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b7e19-fd09-4475-96ac-1728aa84a257 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.686533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.687362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.687652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.687930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.698076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:41.706323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:41.707243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:41.708080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 45/78] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:41.708754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.721859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 34/79] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:41.722564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.763865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.815495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c32dec-b898-4eb6-8f6c-434d7e5aa413 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.818451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:41.821652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfc01d-191a-42fe-832d-528fa54bdef4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.854213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.854903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.855231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.855512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.856283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:41.856595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:41.858012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:41.858784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:41.859956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:41.860240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:41.860512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:41.861719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:41.863201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:41.869677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 35/80] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:41.869677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.869915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 46/81] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:41.869915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:41.973516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274faaf-aac2-46a4-a929-5e06223a4f9e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.976301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68747caa-84ad-41ad-b3cd-102a70240a2a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:41.988030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.012061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.013054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.013519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.013938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.017084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:42.017084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.017780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.018078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.018505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.018786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.019102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.020936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.022802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.024505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 36/82] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.025928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.028910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 47/83] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.029919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.067251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.138301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b322dd3-195d-4454-8b5c-90adb49ce45e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.138780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35546efa-5fb9-4449-b241-686dac7d1b9d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.173980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.174641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.174927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.175245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.180949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.181814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.182095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.182371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.183623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.185067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.187034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 37/84] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.187784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.222190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.251292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.252693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.311824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bcec3-3134-4d6f-ac97-3a9d8d3e79b1 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.315767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.356286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.357960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 48/85] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:42.358091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.358091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.358091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.358267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.362895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.364494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.366214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 38/86] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.366992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.449030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.475663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fdba3-73f9-4102-a253-b85890d0647b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.476723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293b1ec-221c-4ad0-9402-5c83610f1d4b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.516078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.516676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.516970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.517249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.517975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.519474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.521497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.522387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.522675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.522958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.524232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.526046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.528417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 39/87] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.529092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.546769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 49/88] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:38:42 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:42.547661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.582622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.652427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8e494-623f-446f-a41a-0753f0dfc40c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.672968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86c605-20a7-4379-baf8-005df7afdd4c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.690126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.690807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.691114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.691399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.692263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] RBAC: Adding query filter params (role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=d75f48bf9c68429b994a473847b8ebb0) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:42.692584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.694062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.711090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 40/89] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/role_assignments?user.id=d75f48bf9c68429b994a473847b8ebb0&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.711090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.711393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.718944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.720138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.722466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.722466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.726370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:42.729331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:42.731659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 50/90] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:42.732325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:42.870690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c46da1-4eb4-42e0-ac94-dd6cbd74eaf2 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.874317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:42.879252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763683f8-0a56-4820-861b-ebc425870047 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:42.928718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.929367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.929655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.929938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:42.936809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:42.937813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:42.938291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:42.938757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:43.007398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.008889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.034156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:43.042001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.043569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.155938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 41/91] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:43.156803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:43.160869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:43.172016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 51/92] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:43.173162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:43.296181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58f20a-2802-4122-9985-c43b6ebfafdc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:43.304095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:43.309060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dec2ff-c2dd-41fe-970d-c75469f92d59 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:43.354395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:43.355081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:43.355388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:43.355672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:43.356534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=d75f48bf9c68429b994a473847b8ebb0) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:43.356854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.358317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.360604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:43.361537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:43.361871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:43.362151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:43.363570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.365084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.366818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 52/93] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:43 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:43.367617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:43.381587 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 42/94] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:43 2022] GET /identity/v3/role_assignments?user.id=d75f48bf9c68429b994a473847b8ebb0&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:43.382501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:43.567475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:43.596704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20526faa-49de-4a22-b176-4fc1be3c8a36 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:43.635428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:43.636309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:43.636598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:43.636880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] PATH_INFO: `/v3/projects/31236a5c264140f4acb6b5a98747f848` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:43.638091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] RBAC: Authorizing `identity:get_project(project_id=31236a5c264140f4acb6b5a98747f848)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.639587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.641309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 53/95] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:43 2022] GET /identity/v3/projects/31236a5c264140f4acb6b5a98747f848 => generated 312 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:43.648002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:43.666169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303ae2b-3e95-4cd5-83e6-e0b17283b968 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:43.716748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:43.717434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:43.717721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:43.718153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:43.719010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=ec67f4a1435249d49af5a7f7df4f7a39) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:43.719355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.720809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133afd27-1059-495f-934a-590c8bbf8b60 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.731848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 43/96] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:43 2022] GET /identity/v3/role_assignments?user.id=ec67f4a1435249d49af5a7f7df4f7a39&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:43.743591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:43.846445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7b507-95ac-4921-8e36-55c7bf44169e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:43.892984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:43.893724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:43.894016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:43.894296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:43.895186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=31236a5c264140f4acb6b5a98747f848, user.id=d75f48bf9c68429b994a473847b8ebb0) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:43.895508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:43.897005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc78bb5-8acb-45ab-991c-f0520e700235 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:43.905827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 54/97] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:43 2022] GET /identity/v3/role_assignments?user.id=d75f48bf9c68429b994a473847b8ebb0&scope.project.id=31236a5c264140f4acb6b5a98747f848&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:45.316577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139b976-79d8-4863-9c4f-cfdb39be865d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:45.317590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139b976-79d8-4863-9c4f-cfdb39be865d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:45.318083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139b976-79d8-4863-9c4f-cfdb39be865d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:45.319875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 44/98] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:45.322208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f613-2637-415b-870c-2cadd64a4605 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:45.323013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f613-2637-415b-870c-2cadd64a4605 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:45.323527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f613-2637-415b-870c-2cadd64a4605 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:45.325090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 55/99] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:45.330374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:45.330893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:45.331410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:45.340196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:45.340751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:45.341224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:45.361368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:45.362591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:45.363030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:45.364435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:45.481634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 45/100] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:45.488532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 56/101] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:45.518128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:45.531925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:45.575607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:45.643904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:45.719991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594cb2f-a7c4-4328-91b8-889e6573bd75 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:45.723990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b47dc0-098b-447b-b1cd-ff44027c0970 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:45.782588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:45.783557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:45.783844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:45.784122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:45.800779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:45.804302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:45.805133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:45.805423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:45.805703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:45.808991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:45.809841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:38:45.810613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:45.815538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 57/102] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:45.822246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:45.860108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:45.876376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:45.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:45.880623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 46/103] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:38:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:45.896096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:45.950534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:45.966595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466a465-b6ac-4985-b70e-d33839801a61 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.011827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.012901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.013199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.013652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.014442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:46.014762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.016209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.035223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 58/104] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:46.041182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:46.145930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3760c6-121e-4f2a-bd88-8d2db192e6b4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.187530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:46.212823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.213560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.213831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.214081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.214858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:46.215201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.216622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.230993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 47/105] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:38:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:46.237829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:46.257221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a0f00-fbc4-4287-be6b-ac0e9a706a73 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.326580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.327508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.327793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.328096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] PATH_INFO: `/v3/users/243eeb523cd44c81b957f694113f6513` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.356087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] RBAC: Authorizing `identity:get_user(user_id=243eeb523cd44c81b957f694113f6513)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.357574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.359783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 59/106] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:46 2022] GET /identity/v3/users/243eeb523cd44c81b957f694113f6513 => generated 289 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:46.365214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:46.389575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:46.427416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494307b-21f2-4113-8d46-646803e1fe0e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.500319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7503caac-6402-4e9e-97f3-3e08f3e81748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.501552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503caac-6402-4e9e-97f3-3e08f3e81748 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.502027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503caac-6402-4e9e-97f3-3e08f3e81748 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.502516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503caac-6402-4e9e-97f3-3e08f3e81748 None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.508108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503caac-6402-4e9e-97f3-3e08f3e81748 None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.510463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7503caac-6402-4e9e-97f3-3e08f3e81748 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.512399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 48/107] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:46 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:46.513834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9be71-f49d-4c8e-995f-dd8f4fc7010c None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.514355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9be71-f49d-4c8e-995f-dd8f4fc7010c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.514826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9be71-f49d-4c8e-995f-dd8f4fc7010c None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.516290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 49/108] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:46.520124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9be71-f49d-4c8e-995f-dd8f4fc7010c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:46.547395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4472cf5-8bef-424e-b078-2f296c3ccd41 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.592761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5584bbc-aa02-41c2-88b3-b66776f7393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.593443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5584bbc-aa02-41c2-88b3-b66776f7393d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.593731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5584bbc-aa02-41c2-88b3-b66776f7393d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.594011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5584bbc-aa02-41c2-88b3-b66776f7393d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.613170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:46.684918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9be71-f49d-4c8e-995f-dd8f4fc7010c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.704472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5584bbc-aa02-41c2-88b3-b66776f7393d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.706086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5584bbc-aa02-41c2-88b3-b66776f7393d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.716754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:46.725046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaa5b-0143-43c7-b0f0-d66deae3799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.726266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaa5b-0143-43c7-b0f0-d66deae3799a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.726759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaa5b-0143-43c7-b0f0-d66deae3799a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.727242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaa5b-0143-43c7-b0f0-d66deae3799a None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.729503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaa5b-0143-43c7-b0f0-d66deae3799a None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.731233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaa5b-0143-43c7-b0f0-d66deae3799a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.733473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 50/109] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:46 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:46.735270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409914-155f-4c7b-a47f-3801ca25360a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.735559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409914-155f-4c7b-a47f-3801ca25360a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.735841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6409914-155f-4c7b-a47f-3801ca25360a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.754727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c6409914-155f-4c7b-a47f-3801ca25360a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:46.756035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c6409914-155f-4c7b-a47f-3801ca25360a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:46.815100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 60/110] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:38:46.816365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011da9-0d46-4d9d-b2ae-396840358f7c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.816662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011da9-0d46-4d9d-b2ae-396840358f7c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.816943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011da9-0d46-4d9d-b2ae-396840358f7c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.818347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 61/111] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:46.819919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4145-8d3b-4977-b2c6-c532a3fa791c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.819919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4145-8d3b-4977-b2c6-c532a3fa791c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.820112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4145-8d3b-4977-b2c6-c532a3fa791c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.822112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 62/112] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:46.822395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4145-8d3b-4977-b2c6-c532a3fa791c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:46.835284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 51/113] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:46.836738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc460299-83f4-45fd-be0b-a54084c1f62e None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.837581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc460299-83f4-45fd-be0b-a54084c1f62e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.838164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc460299-83f4-45fd-be0b-a54084c1f62e None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.839376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 52/114] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:46.843923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc460299-83f4-45fd-be0b-a54084c1f62e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:46.895161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:46.927646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4145-8d3b-4977-b2c6-c532a3fa791c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.937036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:46.950246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc460299-83f4-45fd-be0b-a54084c1f62e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:46.961763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8af3b-3992-496b-a7a4-2bf719d96d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.963260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8af3b-3992-496b-a7a4-2bf719d96d1a None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.963538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8af3b-3992-496b-a7a4-2bf719d96d1a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.963812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8af3b-3992-496b-a7a4-2bf719d96d1a None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/d75f48bf9c68429b994a473847b8ebb0/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.966044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8af3b-3992-496b-a7a4-2bf719d96d1a None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0857875f64efaa76e5af8b36c50ba, user_id=d75f48bf9c68429b994a473847b8ebb0, role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.967570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8af3b-3992-496b-a7a4-2bf719d96d1a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.981795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 63/115] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:38:46 2022] PUT /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/d75f48bf9c68429b994a473847b8ebb0/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:38:46.983042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db050f4d-a1c6-4705-b39d-717d54d930e2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.983355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db050f4d-a1c6-4705-b39d-717d54d930e2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.983637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db050f4d-a1c6-4705-b39d-717d54d930e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.985728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecb75f-c31f-4c13-9a48-411371680c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:46.986602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecb75f-c31f-4c13-9a48-411371680c22 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.986889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecb75f-c31f-4c13-9a48-411371680c22 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.987215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecb75f-c31f-4c13-9a48-411371680c22 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:46.988495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecb75f-c31f-4c13-9a48-411371680c22 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:46.990015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecb75f-c31f-4c13-9a48-411371680c22 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:46.991896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 53/116] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:46 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:46.993171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:46.993509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:46.993923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.005611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-db050f4d-a1c6-4705-b39d-717d54d930e2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:47.006843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-db050f4d-a1c6-4705-b39d-717d54d930e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:47.012177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:47.013415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:47.089065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 64/117] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:47.090643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.090960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.091342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.098469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 54/118] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:47.104135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.114419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:47.115817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:47.161423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:47.196713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 65/119] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:47.197903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.222759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952f272-882a-4d34-8fed-99749fd8f18c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.233860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:47.262192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.263289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.263749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.264196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.278092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:47.285915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:47.286769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:47.287591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 55/120] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:47.288269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.319895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0d71-c5eb-4c1b-990c-a51f0d2cb18c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.357910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.358626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.358920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.359528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.360385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] RBAC: Adding query filter params (role.id=8ceef45c9f3f430783804f3dfe9ad674, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=243eeb523cd44c81b957f694113f6513) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:47.360705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:47.362193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:47.379089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 66/121] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/role_assignments?user.id=243eeb523cd44c81b957f694113f6513&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=8ceef45c9f3f430783804f3dfe9ad674 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:47.379089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.423838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7c721-73ed-4699-a8c6-29a8cbc6d489 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.436053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:47.471609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.472520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.472811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.473130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.486746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:47.494947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:47.495153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:47.496056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:47.497305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 56/122] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:47.498200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.508671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b20c5e-4d96-4602-9789-36dc0466fe03 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.554276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.555004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.555312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.555585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.559842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:47.562109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:47.564577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 67/123] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:47.565587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.633884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37433f-02a3-45ee-9269-36ba6315451a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.671136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.671967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.672381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.672663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.684726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:47.693030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:47.693917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:47.694704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 57/124] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:47.695402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.696701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df06d-05a4-40ff-8e2d-15ad20d85742 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.733378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:47.746034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.747555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.747839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.748115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.748905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:47.749215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:47.750615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:47.763971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 68/125] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:47.764895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.796481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:47.835875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7368f9-8d91-4b3b-b3b7-199b10da08df None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.882943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.883972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.884450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.884821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.885871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:47.886193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:47.888357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:47.893066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d67764-8462-41a3-a472-f95929b77624 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:47.903142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 58/126] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:47.904364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:47.934256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:47.935389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:47.935877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:47.936390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:47.937553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:47.939358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:47.972873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 69/127] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:38:47 2022] POST /identity/v3/projects => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:47.975359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.069628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:48.101342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73459420-8dfb-4a34-a96b-48668ee50c63 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.163759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:48.186702 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.187819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.188345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.188824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.189799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:48.190307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.191951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.197696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bf4cc-350b-4767-965b-afc886c462d8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.204295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 59/128] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.205485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.247350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.248886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.249397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.250814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.252382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.254117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.257230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 70/129] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:47 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.258227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.347130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a810f3-1c43-4428-a8a1-117dbad9524c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.413267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.414068 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.414349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.414627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.416239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97934f3f-1ee5-4a76-b9a5-fc6ca58b61ab None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.418343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.419845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.421513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 60/130] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:48 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.422277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.469198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:48.482655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.483586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.483873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.484414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.485658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.487156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.489140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 71/131] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:48 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.490212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.515439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:48.619708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd7579-24ce-4d50-8a53-8efbba6bc4fd None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.686493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.687384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.688335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.688630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.689982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.691519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.693299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 61/132] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:48 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.694080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.704901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcc433-b8e9-459d-9818-23a35f1b7c4a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.793177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.794061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.794351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.794633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.799087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.800573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.806170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 72/133] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:48 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.806952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.839303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:48.854301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7bbd-462e-4d92-8037-6a1d41c56d13 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:48.896780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:48.897654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:48.897935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:48.898209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] PATH_INFO: `/v3/projects/31236a5c264140f4acb6b5a98747f848` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:48.903122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] RBAC: Authorizing `identity:get_project(project_id=31236a5c264140f4acb6b5a98747f848)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:48.908203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:48.910475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 62/134] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:48 2022] GET /identity/v3/projects/31236a5c264140f4acb6b5a98747f848 => generated 312 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:48.912034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:48.991409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:49.029399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c18240-d657-4e36-aa57-97ed58cc2564 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:49.070490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:49.072289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.072571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.072847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/d75f48bf9c68429b994a473847b8ebb0/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.081743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0857875f64efaa76e5af8b36c50ba, user_id=d75f48bf9c68429b994a473847b8ebb0, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:49.090718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:49.114798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 73/135] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:38:48 2022] PUT /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/d75f48bf9c68429b994a473847b8ebb0/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:38:49.115807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:49.126003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce84ed-11db-4f3f-8e7f-e310dde0f394 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:49.168620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94daf8a6-e307-4b23-9978-a4ebd208568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:49.170597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daf8a6-e307-4b23-9978-a4ebd208568a None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.171145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daf8a6-e307-4b23-9978-a4ebd208568a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.171651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94daf8a6-e307-4b23-9978-a4ebd208568a None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/ec67f4a1435249d49af5a7f7df4f7a39/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.174144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94daf8a6-e307-4b23-9978-a4ebd208568a None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0857875f64efaa76e5af8b36c50ba, user_id=ec67f4a1435249d49af5a7f7df4f7a39, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:49.176085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94daf8a6-e307-4b23-9978-a4ebd208568a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:49.193447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 63/136] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:38:48 2022] PUT /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/ec67f4a1435249d49af5a7f7df4f7a39/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:38:49.242387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d569-fed3-4a16-a87a-7de632c3f1a2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.243029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d569-fed3-4a16-a87a-7de632c3f1a2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.244083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35d569-fed3-4a16-a87a-7de632c3f1a2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.246055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 64/137] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:49.252011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.252654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.253135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.287122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c6568-fbd9-40df-9cae-5e3f891d15e7 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:49.290238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:49.292133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:49.312599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:49.323163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:49.324322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.324431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.324676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] PATH_INFO: `/v3/projects/31236a5c264140f4acb6b5a98747f848/users/d75f48bf9c68429b994a473847b8ebb0/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.326879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] RBAC: Authorizing `identity:create_grant(project_id=31236a5c264140f4acb6b5a98747f848, user_id=d75f48bf9c68429b994a473847b8ebb0, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:49.328497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:49.343912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:49.350693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 74/138] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:38:49 2022] PUT /identity/v3/projects/31236a5c264140f4acb6b5a98747f848/users/d75f48bf9c68429b994a473847b8ebb0/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:38:49.422928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 65/139] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:49.455211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:49.597556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f90d9-ca03-4b28-8d3f-cb1a282f9ab6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:49.638351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e58e5-d89a-4c21-8bb5-414407f1c5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:49.639208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e58e5-d89a-4c21-8bb5-414407f1c5ce None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.639493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e58e5-d89a-4c21-8bb5-414407f1c5ce None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.639768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e58e5-d89a-4c21-8bb5-414407f1c5ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.657259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e58e5-d89a-4c21-8bb5-414407f1c5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a10e58e5-d89a-4c21-8bb5-414407f1c5ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:49.666349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:49.667308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:49.668099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 75/140] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:38:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:49.673832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:49.823789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ced7525-2275-49cf-8634-87b4967f1c1e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:49.845571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160216bc-9919-4429-a99d-21557e4cfde5 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.845902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160216bc-9919-4429-a99d-21557e4cfde5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.846134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160216bc-9919-4429-a99d-21557e4cfde5 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.847844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 76/141] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:49.854496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.854746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.855013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.866782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:49.867847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:49.868352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:49.868839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:49.869796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:49.870294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:49.872022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:49.881210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 66/142] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:38:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:49.891085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:49.893975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:49.895239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:49.911429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:49.983421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:50.004382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 77/143] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:50.069486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a110c6-bce4-4de4-8ab8-945974beb77f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:50.074553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:50.162309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:50.163121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:50.163406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:50.163676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:50.167942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:50.169431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:50.171284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 67/144] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:49 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:50.176862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:50.298375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a029f2-726d-40cb-8e99-c90e5c6d23da None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:50.319957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:50.358792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b8133-5c8a-4879-8409-0025a39566dd None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:50.390106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:50.394172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:50.394612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:50.395038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:50.416868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:50.433370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:50.434030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:50.434314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:50.434590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:50.434911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:50.435951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:38:50.436912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:50.438955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 78/145] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:38:50.446036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:50.530428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:50.570590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:50.573819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:50.627043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed44571-8962-43f0-a7d7-bae860b1a5b8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:50.674549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:50.675192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:50.675474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:50.675748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:50.678847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] RBAC: Adding query filter params (name=reader) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:50.679194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:50.680687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:50.697972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 79/146] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:50.704407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:50.714733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:50.747255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 68/147] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:38:50.756128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:50.916637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba68359-4bf2-4c27-ac29-bfbe5dd6556d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:50.920700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62131-bdf3-4a49-8a39-ce95e9c5a49e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:50.939567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:50.983915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:50.985069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:50.985347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:50.985619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] PATH_INFO: `/v3/users/243eeb523cd44c81b957f694113f6513` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:50.987719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:50.988580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:50.988861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:50.989137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:50.989897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] RBAC: Authorizing `identity:get_user(user_id=243eeb523cd44c81b957f694113f6513)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:50.991184 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:50.992680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:50.995544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:50.997416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 80/148] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:50 2022] GET /identity/v3/users/243eeb523cd44c81b957f694113f6513 => generated 289 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:51.003660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 69/149] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:50 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:51.004574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:51.025377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:51.097500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:51.185967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9c1bf-7201-4302-a078-5c9a2dd3f4de None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:51.210493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4ec8f-2023-4110-b41a-ba440f4f958a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:51.223366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:51.224239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.224528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.224823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.226051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:51.227645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:51.235082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 70/150] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:51 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:51.235559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:51.276653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:51.277688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.278185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.278739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.279794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] RBAC: Adding query filter params (role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=d75f48bf9c68429b994a473847b8ebb0) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:51.280299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:51.282122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f86094-2b84-4750-ab2f-76fc1f0fab57 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:51.295777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 81/151] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:51 2022] GET /identity/v3/role_assignments?user.id=d75f48bf9c68429b994a473847b8ebb0&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:51.297443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e4b18-d37d-4ccf-8c91-274efa7e3aad None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.297923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e4b18-d37d-4ccf-8c91-274efa7e3aad None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.298486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e4b18-d37d-4ccf-8c91-274efa7e3aad None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.300109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 82/152] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:51.305836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.306419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.306986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.337750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:51.339312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:51.367672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05d920-fee5-4b1d-8aa1-49faea8a3ad6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:51.390589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:51.424222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:51.426795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:51.427801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.428075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.428353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/243eeb523cd44c81b957f694113f6513/roles/8ceef45c9f3f430783804f3dfe9ad674` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.436634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0857875f64efaa76e5af8b36c50ba, user_id=243eeb523cd44c81b957f694113f6513, role_id=8ceef45c9f3f430783804f3dfe9ad674)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:51.438128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:51.458261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 71/153] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:38:51 2022] PUT /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/users/243eeb523cd44c81b957f694113f6513/roles/8ceef45c9f3f430783804f3dfe9ad674 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:38:51.488536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 83/154] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:51.537375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:51.695249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4d970-605f-465f-bffa-350549711538 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:51.737247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd843e-4761-4992-92e7-40c9512b18ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:51.738106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd843e-4761-4992-92e7-40c9512b18ea None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.738437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd843e-4761-4992-92e7-40c9512b18ea None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.738769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd843e-4761-4992-92e7-40c9512b18ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.739990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd843e-4761-4992-92e7-40c9512b18ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:51.741527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd843e-4761-4992-92e7-40c9512b18ea None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:51.743411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 72/155] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:51.749095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:51.894044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e660e-adf8-48a4-8b36-10ecd3eae6b9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:51.935160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3306b-b00e-4a6c-a535-1999ec13e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:51.935840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3306b-b00e-4a6c-a535-1999ec13e48c None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:51.936154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3306b-b00e-4a6c-a535-1999ec13e48c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:51.936387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3306b-b00e-4a6c-a535-1999ec13e48c None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:51.937114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3306b-b00e-4a6c-a535-1999ec13e48c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:51.938559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3306b-b00e-4a6c-a535-1999ec13e48c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:51.965836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 84/156] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:38:51 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:52.128397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2457697-1c8b-4184-8a52-4636f58b928e None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.129038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2457697-1c8b-4184-8a52-4636f58b928e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.129515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2457697-1c8b-4184-8a52-4636f58b928e None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.131124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 73/157] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:52.137452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.137774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.138031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.159272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:52.160473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:52.190189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:52.237015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5286b-eaf6-4d5c-ba92-014b0f4f5ff1 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.237338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5286b-eaf6-4d5c-ba92-014b0f4f5ff1 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.237577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5286b-eaf6-4d5c-ba92-014b0f4f5ff1 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.239018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 74/158] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:52.245553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163d8c-643f-4278-8de5-b56e4de54669 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.246163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163d8c-643f-4278-8de5-b56e4de54669 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.246638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163d8c-643f-4278-8de5-b56e4de54669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.264958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 85/159] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:52.276713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9c163d8c-643f-4278-8de5-b56e4de54669 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:52.277961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9c163d8c-643f-4278-8de5-b56e4de54669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:52.300902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.328262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:52.378984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 75/160] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:52.380339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ad7fa-ef2a-413e-b399-b40fdc4786c3 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.380632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ad7fa-ef2a-413e-b399-b40fdc4786c3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.380907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ad7fa-ef2a-413e-b399-b40fdc4786c3 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.382643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 76/161] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:52.390283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.390764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.391651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.417837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:52.419824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:52.444774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748353bc-6615-4f31-b5bb-177e268ccb39 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:52.483560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:52.484965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.485254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.485533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.498551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:52.507438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:52.508389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:52.509411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 86/162] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:38:52.510148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.518829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 77/163] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:52.520103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.539041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:52.627000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5b22b-edc9-44a2-a43e-d77fca9878c7 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:52.637471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:52.640741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da674aa-f436-41d1-8c04-0021d71cd771 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:52.667569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:52.668387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.668666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.668940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.684466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:52.687789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:52.688599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.688962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.689338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.690270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:52.690635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:52.692083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:52.692304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:52.693163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:52.693948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:52.699095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 87/164] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:52.699969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.707942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 78/165] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:52.708839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.835436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:52.840623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e664-4c08-4be6-abb3-24a8ed0e1e15 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:52.863539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cae6d-559f-4654-a86a-e81a2d7b1c12 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:52.886873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:52.887648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.888445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.888730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.889780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:52.890097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:52.892215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:52.905700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 79/166] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:52.907431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:52.907656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.908250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:52.908533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:52.908961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:52.926182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:52.938058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:52.938999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:52.939853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:52.940464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 88/167] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:52.941096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:52.941359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.033944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74905e-2fb2-4b69-a418-7e8c688d7708 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.062358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7426c-8228-4fee-a0a3-eacde8fa7a92 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.072723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.073688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.074023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.074310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] PATH_INFO: `/v3/users/ec67f4a1435249d49af5a7f7df4f7a39` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.075691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] RBAC: Authorizing `identity:get_user(user_id=ec67f4a1435249d49af5a7f7df4f7a39)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.077187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.079301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 80/168] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/users/ec67f4a1435249d49af5a7f7df4f7a39 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:53.080115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.106437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.107364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.107649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.108010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.109172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.110896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.112748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 89/169] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:52 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:53.115902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.127087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.186038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.196366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ad45e-c92e-4539-8927-997c45bfa90a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.231781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeaaf2-95b3-4d36-83c7-bd39d9342fd6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.247649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.250895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.251377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.251827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.252762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:53.254401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.257901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.279847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 81/170] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:53.280987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.299597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.300601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.301073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.301535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.408318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.410112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.419083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.470089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56632e4-9633-4a8c-8aef-02a7b245a448 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.517040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 90/171] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:53.518176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.536898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.537672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.537850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.538117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.590642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.623258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caeb59f-6f2c-4c5f-aa4c-fb5f1c5b073c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.631046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.659768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.660985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.661445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.661906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] PATH_INFO: `/v3/users/d75f48bf9c68429b994a473847b8ebb0` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.663359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] RBAC: Authorizing `identity:get_user(user_id=d75f48bf9c68429b994a473847b8ebb0)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.665064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.667012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 91/172] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/users/d75f48bf9c68429b994a473847b8ebb0 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:53.668002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.670409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.675755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.775361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f93680-3c40-420b-b60e-d9ef98174c2d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.812799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.814020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.814488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.814943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.816395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:53.818155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:53.820108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 92/173] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:53.821057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.866986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:53.870143 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 82/174] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:53.871013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:53.928647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b99e4-8dae-45f4-aa38-4c6b6269830a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:53.965812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:53.966805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:53.967297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:53.967926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:53.986095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:54.042844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:54.044457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:54.047851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf9570-358a-4346-a6c0-22392ba8624d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:54.100303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:54.100967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.101244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.101511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.102348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=ec67f4a1435249d49af5a7f7df4f7a39) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:54.102855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:54.104565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:54.117470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 83/175] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/role_assignments?user.id=ec67f4a1435249d49af5a7f7df4f7a39&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:54.118326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:54.129909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:54.237075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 93/176] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:54.242672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:54.346666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d896c22-18ca-488d-9b18-4d2ba7920fbe None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:54.383347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ea33d-c631-49e2-8898-1bdf7642121d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:54.400217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:54.438380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:54.439284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.439562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.439831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] PATH_INFO: `/v3/projects/31236a5c264140f4acb6b5a98747f848` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.441195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] RBAC: Authorizing `identity:get_project(project_id=31236a5c264140f4acb6b5a98747f848)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:54.442665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:54.446603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 84/177] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:54 2022] GET /identity/v3/projects/31236a5c264140f4acb6b5a98747f848 => generated 312 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:54.458421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:54.459321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.459602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.459875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.464074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:54.468399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:54.474005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:54.476093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 94/178] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:54 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:54.482229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:54.502742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:54.662280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604036e-6e12-4efa-b6c5-e35621c18b65 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:54.696730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177273a-f32a-45ff-a2f3-29340f7249c1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:54.713324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:54.713985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.714263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.714533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.717173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=31236a5c264140f4acb6b5a98747f848, user.id=d75f48bf9c68429b994a473847b8ebb0) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:54.717524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:54.718940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee1096-3c40-4c27-8da9-c3ca5196107c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:54.731483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 85/179] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:54 2022] GET /identity/v3/role_assignments?user.id=d75f48bf9c68429b994a473847b8ebb0&scope.project.id=31236a5c264140f4acb6b5a98747f848&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:54.733135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabbf1-8cda-4df0-b7a2-c56dc5e1f0e0 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.733658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabbf1-8cda-4df0-b7a2-c56dc5e1f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.734133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabbf1-8cda-4df0-b7a2-c56dc5e1f0e0 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.743692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 86/180] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:54.749072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.749624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.750102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.753107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:54.753828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:54.754110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:54.754615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:54.756301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=d75f48bf9c68429b994a473847b8ebb0) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:54.757036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:54.759689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:54.768940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 95/181] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:54 2022] GET /identity/v3/role_assignments?user.id=d75f48bf9c68429b994a473847b8ebb0&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:54.780317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:54.781910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:54.866895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:54.895260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 87/182] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:54.921120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.037736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:55.064395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aeb46-c43e-4b91-a89e-602aa72bb52c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.100732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:55.101533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.101808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.102077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.112933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:55.121385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:55.122296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:55.123153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 96/183] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:55.130015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.253029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487109e5-7fd6-4532-8de0-d61c64dd64ba None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.290639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:55.291341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.291619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.291931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.292690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:55.293039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:55.294366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:55.303076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 88/184] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:55.312173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.420311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e94f29-8199-479c-bf06-2483f5432460 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.455456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177516bb-285f-4265-958f-ef24df842f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:55.456385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177516bb-285f-4265-958f-ef24df842f41 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.456667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177516bb-285f-4265-958f-ef24df842f41 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.456941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177516bb-285f-4265-958f-ef24df842f41 None admin] PATH_INFO: `/v3/users/243eeb523cd44c81b957f694113f6513` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.458142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177516bb-285f-4265-958f-ef24df842f41 None admin] RBAC: Authorizing `identity:get_user(user_id=243eeb523cd44c81b957f694113f6513)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:55.459656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177516bb-285f-4265-958f-ef24df842f41 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:55.461440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 97/185] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/users/243eeb523cd44c81b957f694113f6513 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:55.467294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.515156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a295fb-e494-4321-aada-65de90ea51e6 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.515397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a295fb-e494-4321-aada-65de90ea51e6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.515645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a295fb-e494-4321-aada-65de90ea51e6 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.517032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 98/186] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:55.517618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:55.522102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.522394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.522676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.527911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:55.540691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:55.542261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:55.584051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fdb3f-c755-4e13-b953-6dceec6a0873 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.616603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 99/187] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:55.629458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:55.630116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.630394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.630667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.641093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.732879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:55.735049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:55.748471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b23b07-54ea-44eb-84d7-e0b593c4afc4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.781794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:55.787365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:55.788165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.788443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.788719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.800332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:55.801553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:55.809401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:55.810309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:38:55.811156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 100/188] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:38:55.818269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.866895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 89/189] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:55.872458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:55.933338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4aee84-817a-4903-86b7-b2ea91baebbc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.969015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:55.969690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.969977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.970254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.971006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:55.971355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:55.972524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be52878-c709-4e2c-b803-4b26060bf1bc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:55.972978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ee3b2-514f-47a2-90c5-8bcaebad0d9a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:55.981619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 101/190] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:55.984197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec00d6-70de-43d7-a701-4bf4c261636b None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.984814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec00d6-70de-43d7-a701-4bf4c261636b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.985163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec00d6-70de-43d7-a701-4bf4c261636b None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.986604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 102/191] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:55.990788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9840108-1470-4e0e-9ef1-c27b0cfba67a None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:55.990788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9840108-1470-4e0e-9ef1-c27b0cfba67a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:55.990788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9840108-1470-4e0e-9ef1-c27b0cfba67a None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:55.990788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 103/192] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:55.990788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9840108-1470-4e0e-9ef1-c27b0cfba67a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.013048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fd68c-a580-4c97-b130-5d70ebb9a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.013905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fd68c-a580-4c97-b130-5d70ebb9a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.014182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fd68c-a580-4c97-b130-5d70ebb9a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.014451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fd68c-a580-4c97-b130-5d70ebb9a4b7 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.015749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fd68c-a580-4c97-b130-5d70ebb9a4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.017251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fd68c-a580-4c97-b130-5d70ebb9a4b7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.021875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 90/193] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:56.021875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc9bf-9bc5-4a9e-a7ff-5c0f2bce3831 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.021875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc9bf-9bc5-4a9e-a7ff-5c0f2bce3831 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.021875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc9bf-9bc5-4a9e-a7ff-5c0f2bce3831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.036559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3b5fc9bf-9bc5-4a9e-a7ff-5c0f2bce3831 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:56.037799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3b5fc9bf-9bc5-4a9e-a7ff-5c0f2bce3831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:56.040691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.057692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.106488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 91/194] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:56.107354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9840108-1470-4e0e-9ef1-c27b0cfba67a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.107807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.108097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.108376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.125026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:56.126213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:56.143897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.144773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.145054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.145325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] PATH_INFO: `/v3/users/825464a649f24070aa55b6b968e92f5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.158894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] RBAC: Authorizing `identity:get_user(user_id=825464a649f24070aa55b6b968e92f5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.160391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.162069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 104/195] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:55 2022] GET /identity/v3/users/825464a649f24070aa55b6b968e92f5c => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:56.162814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.193301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 92/196] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:56.195301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.262559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9cca63-961b-410c-a88a-3b2b0e900bda None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.264012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.276672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.295281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab2b9a-af5d-44ab-83a6-e4b5e161d8cd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.297728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.298397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.298681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.298959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.299827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] RBAC: Adding query filter params (role.id=8ceef45c9f3f430783804f3dfe9ad674, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba, user.id=243eeb523cd44c81b957f694113f6513) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:56.300144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.301600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.307467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 105/197] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/role_assignments?user.id=243eeb523cd44c81b957f694113f6513&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=8ceef45c9f3f430783804f3dfe9ad674 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:56.308253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.334384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.335348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.335700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.336049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.337505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.339311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.341325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 93/198] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:56.346692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.422927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d0e1e-e400-41e7-b0d1-80de123e32c0 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.456906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f139a-be96-412d-85f1-de5c9d54fdd0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.460057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.460778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.461161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.462170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.500773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.501598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.501877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.502146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.513055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.517169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:56.519968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:56.526984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:56.527932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:56.528737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 94/199] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:56.529408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.549545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.550990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.629934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf36a38-a41d-42c8-a648-96ead7f8c1b8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.650761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 106/200] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:38:56.655444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.668050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.668724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.669004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.669279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.669994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.671470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.692187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 95/201] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:38:56 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:56.692187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.737018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.767394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:56.770594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf15d8e-10dd-4770-9b55-09b59b0527c5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.814177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.814777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.815088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.815367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.816138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:56.816448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.817895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.824651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29639a0-cbfd-4605-b53d-f1278d0e240c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.827824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 107/202] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:56.833419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.864677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.865546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.865828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:56.866105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:56.867358 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:56.869372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:56.871335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 96/203] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:56.876954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:56.952387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106a6b9-cd95-4c05-a692-2dfce62f3e18 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:56.997606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:56.998883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:56.999615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:57.000044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] PATH_INFO: `/v3/users/825464a649f24070aa55b6b968e92f5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:57.001816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] RBAC: Authorizing `identity:get_user(user_id=825464a649f24070aa55b6b968e92f5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:57.004030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:57.005097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28ac90-87d6-416b-a365-c070135e1e7e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:57.006317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 108/204] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/users/825464a649f24070aa55b6b968e92f5c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:57.012206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:57.024434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:57.032743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:57.047715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:57.049015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:57.049015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:57.049015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:57.050268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=825464a649f24070aa55b6b968e92f5c) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:57.050268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:57.051790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:57.058900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 97/205] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:56 2022] GET /identity/v3/role_assignments?user.id=825464a649f24070aa55b6b968e92f5c&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:38:57.150647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19f72d-9472-4f46-a534-feb720c6788d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:57.189621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:57.190549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:57.190979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:57.191439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:57.268447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:57.269947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:57.278067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:57.376442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 109/206] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:38:57.384210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:57.490166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fd552-9dee-409c-9727-0e762ac1652d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:57.526314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21d21f-1caa-4969-a21a-ef43d80318f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:57.527218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21d21f-1caa-4969-a21a-ef43d80318f9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:57.527503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21d21f-1caa-4969-a21a-ef43d80318f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:57.527780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21d21f-1caa-4969-a21a-ef43d80318f9 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:57.529116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21d21f-1caa-4969-a21a-ef43d80318f9 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:57.530632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21d21f-1caa-4969-a21a-ef43d80318f9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:57.532700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 98/207] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:38:57 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:57.538565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:57.656112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a83ad-7a5e-4981-bbb4-c098cded9bdf None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:57.667645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:57.697994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:57.698663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:57.698942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:57.699246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:57.700090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] RBAC: Adding query filter params (role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=825464a649f24070aa55b6b968e92f5c) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:57.700401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:57.701912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54657783-26ea-48a3-b1a2-f2e7712363d4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:57.708185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 110/208] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:38:57 2022] GET /identity/v3/role_assignments?user.id=825464a649f24070aa55b6b968e92f5c&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:58.903710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb61138-7b42-4c62-83f9-ac3a3bb46ab3 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:58.904261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb61138-7b42-4c62-83f9-ac3a3bb46ab3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:58.904261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb61138-7b42-4c62-83f9-ac3a3bb46ab3 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:58.905646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 99/209] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:58.911109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd57717-b077-41ea-a55f-a287bc360a65 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:58.911582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd57717-b077-41ea-a55f-a287bc360a65 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:58.911751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd57717-b077-41ea-a55f-a287bc360a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:58.929771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3fd57717-b077-41ea-a55f-a287bc360a65 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:58.932151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3fd57717-b077-41ea-a55f-a287bc360a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:59.004361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 111/210] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:38:59.029315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb61138-7b42-4c62-83f9-ac3a3bb46ab3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.099405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.100710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cbe59-fd39-4ac3-96cb-f0fad162d793 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.101091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cbe59-fd39-4ac3-96cb-f0fad162d793 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.101444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cbe59-fd39-4ac3-96cb-f0fad162d793 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.103637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 112/211] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:38:59.108980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b113dd8-47b2-44d6-9fb1-3c3ba82ec01d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.109513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b113dd8-47b2-44d6-9fb1-3c3ba82ec01d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.109805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b113dd8-47b2-44d6-9fb1-3c3ba82ec01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.126194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4b113dd8-47b2-44d6-9fb1-3c3ba82ec01d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:59.127416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4b113dd8-47b2-44d6-9fb1-3c3ba82ec01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:59.158976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb61138-7b42-4c62-83f9-ac3a3bb46ab3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.193618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.194417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.194705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.194988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.199704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 113/212] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:59.200862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7598a28-e1c8-405c-8507-0cbf05942cd3 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.201152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7598a28-e1c8-405c-8507-0cbf05942cd3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.201428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7598a28-e1c8-405c-8507-0cbf05942cd3 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.202830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 114/213] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:59.207984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.208270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.208552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.214734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:59.224503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:59.225420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:38:59.226220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:59.226818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:38:59.228036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:38:59.229028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 100/214] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:59.229957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.253992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.301943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 115/215] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:38:59.303029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.353291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efba36-29ba-4fe1-aa44-474d903d131a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.368136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.392181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.392981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.393264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.393546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.394827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:59.396420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:59.398303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 101/216] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:59.403618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.415947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84917dc-d6db-4d80-9fdf-5b4f45c1efa4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.450844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.451545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.451824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.452096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.452843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:59.453147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:59.454538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:59.467742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 116/217] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:59.467742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.491034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.519914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11750e-8765-44da-9412-a6ede9d62e1d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.558269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.559427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.559968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.560390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.570338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c6432-592a-494e-99a4-7e59242558d9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.572590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:38:59.580826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:38:59.581917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:38:59.582866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:38:59.584844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 102/218] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:38:59.585776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.612667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.613335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.613659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.613885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.614593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:59.616058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:59.633551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.639513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 117/219] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:38:59 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:38:59.640557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.718754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed6d6a-d754-4ae0-8190-be5f3b7db909 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.767559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.769271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.771489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.772496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.772847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] PATH_INFO: `/v3/users/76bcc38a552742c2b8a41c19546ec56e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.789561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] RBAC: Authorizing `identity:get_user(user_id=76bcc38a552742c2b8a41c19546ec56e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:59.791199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:59.792983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 103/220] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/users/76bcc38a552742c2b8a41c19546ec56e => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:59.798504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.802940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b7624-95ed-452a-b4cc-857772f4f25c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.853427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.854098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.854378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.854653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.855446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:38:59.855753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:38:59.857225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9bb285-49e9-445b-8137-b54c79ec9f46 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:38:59.868766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 118/221] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:38:59.870056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db94e26-8e26-4604-8259-1077025e9407 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.870720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db94e26-8e26-4604-8259-1077025e9407 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.870720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db94e26-8e26-4604-8259-1077025e9407 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:38:59.872045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 119/222] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:38:59.876352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db94e26-8e26-4604-8259-1077025e9407 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:38:59.926472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:38:59.930975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30d73-8edb-4096-b5bf-f6e660ab9024 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:38:59.976874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:38:59.979020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:38:59.980112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:38:59.980112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.014165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db94e26-8e26-4604-8259-1077025e9407 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.050333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b1d75-98c8-405b-aaf2-bc9f9f1e2f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.051249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b1d75-98c8-405b-aaf2-bc9f9f1e2f32 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.051534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b1d75-98c8-405b-aaf2-bc9f9f1e2f32 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.051815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b1d75-98c8-405b-aaf2-bc9f9f1e2f32 None admin] PATH_INFO: `/v3/users/825464a649f24070aa55b6b968e92f5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.053111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b1d75-98c8-405b-aaf2-bc9f9f1e2f32 None admin] RBAC: Authorizing `identity:get_user(user_id=825464a649f24070aa55b6b968e92f5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.054631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b1d75-98c8-405b-aaf2-bc9f9f1e2f32 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.056366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 120/223] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/users/825464a649f24070aa55b6b968e92f5c => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:00.057591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.057926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.058208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.078686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.080073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.087497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:00.088959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:00.099402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:00.182203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 121/224] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:00.183541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.213085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:00.217692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 104/225] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:39:00.223183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.296696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfc0bd-636d-4337-8b26-714ff6a97d5d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.334854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729c446-4a20-4e07-bc92-1af8981ce7dc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.349596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.351509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.352127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.352767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.354449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.358380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.360644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 122/226] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:00.361222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.371007 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:00.382230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.384435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.385154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.385660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.401823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:00.412510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:00.413426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:00.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:00.421341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 105/227] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:00.421341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.486518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d636-e014-40c2-9290-c93ef3098aff None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.497325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:00.527648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.529003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.529538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.530012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.531775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.533626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.535838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 123/228] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:00.536828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.547246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454ebc4-2c28-45c5-9a92-0fe8db5acfcc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.584907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.585815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.586100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.586378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88/users/825464a649f24070aa55b6b968e92f5c/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.588871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=94662f66ac214075b6277e602fdb8c88, user_id=825464a649f24070aa55b6b968e92f5c, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.590217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.604597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 106/229] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:00 2022] PUT /identity/v3/projects/94662f66ac214075b6277e602fdb8c88/users/825464a649f24070aa55b6b968e92f5c/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:00.605466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.626639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:00.658310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7edc-bab5-4368-bc61-836a35408bf4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.701194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.702441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.702936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.703556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.704534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:00.705045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.706821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.717211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 124/230] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:00.721357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91798d9-696b-459d-87be-28785526bbe4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.723537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:00.743756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:00.759908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.760582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.760926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.761508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.762126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=76bcc38a552742c2b8a41c19546ec56e) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:00.762448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.763974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.770853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 107/231] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/role_assignments?user.id=76bcc38a552742c2b8a41c19546ec56e&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:00.849794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e0618-90d6-42bd-8ca3-dfb0311d61b9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:00.890574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:00.891949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:00.892454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:00.892965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] PATH_INFO: `/v3/users/825464a649f24070aa55b6b968e92f5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:00.894678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] RBAC: Authorizing `identity:get_user(user_id=825464a649f24070aa55b6b968e92f5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:00.896518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:00.898771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 125/232] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/users/825464a649f24070aa55b6b968e92f5c => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:00.905834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:01.026334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7d50-4254-4830-9ba5-7f05bf1e1545 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:01.079456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e28468-6a84-4937-8748-41da7b2c760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:01.080756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e28468-6a84-4937-8748-41da7b2c760a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:01.081242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e28468-6a84-4937-8748-41da7b2c760a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:01.081715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e28468-6a84-4937-8748-41da7b2c760a None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:01.083649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e28468-6a84-4937-8748-41da7b2c760a None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:01.085362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e28468-6a84-4937-8748-41da7b2c760a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:01.088724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 108/233] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:00 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:01.094620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:01.098602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:01.206513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d553-f5b6-443a-b226-a23df9f10f87 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:01.242248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aefb35e-8e71-40f0-8c17-fd83ca9817a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:01.243381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb35e-8e71-40f0-8c17-fd83ca9817a3 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:01.243692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb35e-8e71-40f0-8c17-fd83ca9817a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:01.243976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aefb35e-8e71-40f0-8c17-fd83ca9817a3 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88/users/825464a649f24070aa55b6b968e92f5c/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:01.246365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefb35e-8e71-40f0-8c17-fd83ca9817a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=94662f66ac214075b6277e602fdb8c88, user_id=825464a649f24070aa55b6b968e92f5c, role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:01.247951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aefb35e-8e71-40f0-8c17-fd83ca9817a3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:01.262491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 126/234] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:01 2022] PUT /identity/v3/projects/94662f66ac214075b6277e602fdb8c88/users/825464a649f24070aa55b6b968e92f5c/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:01.983303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb13b5a-8285-4aa7-912e-faf0073fdf6c None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:01.983701 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb13b5a-8285-4aa7-912e-faf0073fdf6c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:01.983879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb13b5a-8285-4aa7-912e-faf0073fdf6c None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:01.985510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 109/235] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:01.990917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:01.991242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:01.991527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.009528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:02.010726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:02.098811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:02.105640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 127/236] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:02.154898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb13b5a-8285-4aa7-912e-faf0073fdf6c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:02.229004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:02.303856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb13b5a-8285-4aa7-912e-faf0073fdf6c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:02.351791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35756cd-458a-48da-bdb7-ce3f7586af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:02.353412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35756cd-458a-48da-bdb7-ce3f7586af40 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:02.353743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35756cd-458a-48da-bdb7-ce3f7586af40 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:02.353973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35756cd-458a-48da-bdb7-ce3f7586af40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.368967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35756cd-458a-48da-bdb7-ce3f7586af40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-c35756cd-458a-48da-bdb7-ce3f7586af40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:02.376810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:02.377709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:02.378534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 110/237] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:02.388253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:02.404841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9a4e6-2be4-4eb4-9cd7-8a272630f085 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:02.405251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9a4e6-2be4-4eb4-9cd7-8a272630f085 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:02.410897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9a4e6-2be4-4eb4-9cd7-8a272630f085 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.410897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 111/238] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:39:02.412334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:02.412444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:02.412714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.431081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:02.432372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:02.506087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 112/239] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:02.573620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:02.609663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083b0d0-1449-43b3-b2da-8303acd9e50b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:02.652821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:02.654894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:02.655475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:02.656328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:02.656803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.658466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:02.659185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:02.663132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:02.683953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 128/240] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:02.692027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:02.743831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91845d2a-74bd-4cb6-b4d8-2da69202cc6d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:02.782005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:02.782643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:02.782923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:02.783225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.786860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:02.788344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:02.790626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 113/241] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:02.797870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:02.803961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:02.894754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e77a1-dba7-4809-8bb9-aa8abdf60511 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:02.964963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:02.965845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:02.966127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:02.966400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] PATH_INFO: `/v3/users/7afde253a05444809e8b1b1ce48ecc64` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:02.996733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd36bc-d5a8-47c0-ad39-3f85658c3432 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:02.999532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=7afde253a05444809e8b1b1ce48ecc64)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.001031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.003466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 129/242] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:02 2022] GET /identity/v3/users/7afde253a05444809e8b1b1ce48ecc64 => generated 297 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:03.013412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.018904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:03.046832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334456ef-a6a3-497e-a649-74e2fdfd7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.056129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334456ef-a6a3-497e-a649-74e2fdfd7216 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.056312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334456ef-a6a3-497e-a649-74e2fdfd7216 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.056497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334456ef-a6a3-497e-a649-74e2fdfd7216 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.057281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334456ef-a6a3-497e-a649-74e2fdfd7216 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.058727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334456ef-a6a3-497e-a649-74e2fdfd7216 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.095255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 114/243] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:02 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:03.159202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73369774-0f53-4b46-b5e0-108c2233b922 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.159801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73369774-0f53-4b46-b5e0-108c2233b922 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.159801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73369774-0f53-4b46-b5e0-108c2233b922 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.161407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 115/244] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:39:03.166873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.167196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.167480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.175049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae1480-bc1a-4c34-b752-2f1f46ec2fb2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:03.193052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:03.208890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:03.210191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:03.229798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.230481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.230767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.231048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.317378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 116/245] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:03.342680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.350159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.351703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.471131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:03.481769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989fab1-4548-46ec-b022-1fdda646196e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:03.487368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 130/246] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:39:03.496110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.512867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:03.542981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e0ef5-a925-4144-b9d4-e6041d245e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.543853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0ef5-a925-4144-b9d4-e6041d245e0b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.544128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0ef5-a925-4144-b9d4-e6041d245e0b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.544396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e0ef5-a925-4144-b9d4-e6041d245e0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.562761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e0ef5-a925-4144-b9d4-e6041d245e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-e41e0ef5-a925-4144-b9d4-e6041d245e0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:03.573964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:03.574961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:03.575802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 117/247] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:03.576497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d7f0-6357-4db5-b72e-7090842b6948 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.576749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d7f0-6357-4db5-b72e-7090842b6948 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.577194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d7f0-6357-4db5-b72e-7090842b6948 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.580788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 118/248] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:03.584318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21d7f0-6357-4db5-b72e-7090842b6948 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.631450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c585376-6d19-44a7-9aa9-b1529fee94d3 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:03.674951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c6565-77e2-480f-bcea-8929df5e57e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.675950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c6565-77e2-480f-bcea-8929df5e57e5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.676239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c6565-77e2-480f-bcea-8929df5e57e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.676522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c6565-77e2-480f-bcea-8929df5e57e5 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.677734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c6565-77e2-480f-bcea-8929df5e57e5 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.679233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c6565-77e2-480f-bcea-8929df5e57e5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.681787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 131/249] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:03.684962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.685321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.685669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.707509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:03.708663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:03.717009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21d7f0-6357-4db5-b72e-7090842b6948 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:03.759549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.760196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.760573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.760901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.761836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:03.762151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.763594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.773625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 119/250] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:03.775185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.798470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 132/251] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:03.799409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.808259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:03.825927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:03.922656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adcc41-e4a8-4fe8-8141-57ed8aa5cd19 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:03.930201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f92dd-f871-4431-a7d2-58624eb9a108 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:03.969894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.970130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:03.970564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.970844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.970945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.971241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.971342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.971514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] PATH_INFO: `/v3/users/76bcc38a552742c2b8a41c19546ec56e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.972156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] RBAC: Adding query filter params (role.id=8ceef45c9f3f430783804f3dfe9ad674, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=7afde253a05444809e8b1b1ce48ecc64) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:03.972472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.972727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] RBAC: Authorizing `identity:get_user(user_id=76bcc38a552742c2b8a41c19546ec56e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:03.973921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef05023-79fd-45c8-a675-53b5daa4ca98 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.974161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:03.975964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 133/252] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/users/76bcc38a552742c2b8a41c19546ec56e => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:03.976851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:03.982954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 120/253] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/role_assignments?user.id=7afde253a05444809e8b1b1ce48ecc64&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=8ceef45c9f3f430783804f3dfe9ad674 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:03.984671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9424c-3f40-4959-83cb-8bdd2d34868d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:03.984960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9424c-3f40-4959-83cb-8bdd2d34868d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:03.985241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9424c-3f40-4959-83cb-8bdd2d34868d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:03.987224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 121/254] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:03.988060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9424c-3f40-4959-83cb-8bdd2d34868d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.100643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:04.140249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9424c-3f40-4959-83cb-8bdd2d34868d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.181560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:04.196999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64928fb0-efa1-41c7-a151-3dec3321a0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.197887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64928fb0-efa1-41c7-a151-3dec3321a0e4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.198338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64928fb0-efa1-41c7-a151-3dec3321a0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.198618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64928fb0-efa1-41c7-a151-3dec3321a0e4 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.199884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64928fb0-efa1-41c7-a151-3dec3321a0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:04.201341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64928fb0-efa1-41c7-a151-3dec3321a0e4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:04.203093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 122/255] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:04.204569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.205128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.205410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.225017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:04.226224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:04.237053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f58a19-59c1-42fe-8d72-0db623de730e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.311475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.312292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.312571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.312843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.325012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:04.333272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:04.334161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:04.334967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 134/256] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:04.335685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.380624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 123/257] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:04.381325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.452560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae007508-8848-48d9-8e57-8713e6534f00 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.500415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.501350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.501627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.501901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88/users/76bcc38a552742c2b8a41c19546ec56e/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.504274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] RBAC: Authorizing `identity:create_grant(project_id=94662f66ac214075b6277e602fdb8c88, user_id=76bcc38a552742c2b8a41c19546ec56e, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:04.505757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:04.516845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:04.526895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 135/258] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:04 2022] PUT /identity/v3/projects/94662f66ac214075b6277e602fdb8c88/users/76bcc38a552742c2b8a41c19546ec56e/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:04.528001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.533741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:04.555155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6106bea-f869-4121-b5b9-021313803f6e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.593909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.594560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.594849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.595155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.595922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:04.596288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:04.597965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:04.606725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 124/259] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:04.612803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.656181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5672403-8959-4fe5-81b5-9582d583cce1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.695249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.697079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.697569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.698031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.711219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:04.721107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:04.722016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:04.722897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:04.724327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eee64-d5cc-4fa8-88a9-78ebe0d819ff None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.726033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 136/260] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:04.732034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.769525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.771008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.771515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.772392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] PATH_INFO: `/v3/users/825464a649f24070aa55b6b968e92f5c` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.774802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] RBAC: Authorizing `identity:get_user(user_id=825464a649f24070aa55b6b968e92f5c)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:04.777499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:04.780123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 125/261] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:04 2022] GET /identity/v3/users/825464a649f24070aa55b6b968e92f5c => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:04.785728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.799979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:04.815216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:04.864196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8919-a163-4351-814a-a8a956312d1e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.899732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1aef9-088c-46db-b752-af1a93677d9f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:04.902529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.903402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.903823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.904233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:04.905161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:04.905605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:04.907175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:04.916992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 137/262] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:04.922517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:04.937792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:04.938477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:04.938718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:04.938991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:05.011869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:05.013389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:05.026690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:05.040374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc84df6-646e-408a-bca6-73c5fa5cba4b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:05.107429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:05.119708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:05.120525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:05.120810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:05.121087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] PATH_INFO: `/v3/users/825464a649f24070aa55b6b968e92f5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:05.123432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] RBAC: Authorizing `identity:get_user(user_id=825464a649f24070aa55b6b968e92f5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:05.124905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:05.135646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 138/263] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:04 2022] GET /identity/v3/users/825464a649f24070aa55b6b968e92f5c => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:05.136549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:05.195663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 126/264] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:05.197731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:05.366555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db505dee-78f3-4e00-879c-62f29cbc5bad None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:05.424544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaedc4-b32d-496a-b894-053db1eb1d11 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:05.446835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:05.450380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:05.451054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:05.451362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:05.451646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:05.506532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:05.507725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:05.508068 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:05.508342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:05.509813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:05.511335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:05.519961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 127/265] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:05 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:05.529821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:05.575425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:05.597800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:05.599469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:05.793245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760b474e-e022-4a29-b02b-4a4752926ff3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:05.794113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 139/266] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:05.801701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:05.892365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:05.893036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:05.893317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:05.893588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:05.894436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=825464a649f24070aa55b6b968e92f5c) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:05.894748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:05.896255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:05.909407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 128/267] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:05 2022] GET /identity/v3/role_assignments?user.id=825464a649f24070aa55b6b968e92f5c&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:06.002332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:06.015315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45566810-3e48-4fa8-9c6f-b30c0a5d06d5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:06.077597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dcaeec-8c8f-4cf5-b4d9-c5dfa3a54308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:06.077597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcaeec-8c8f-4cf5-b4d9-c5dfa3a54308 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.077857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcaeec-8c8f-4cf5-b4d9-c5dfa3a54308 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.078094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dcaeec-8c8f-4cf5-b4d9-c5dfa3a54308 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.079419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcaeec-8c8f-4cf5-b4d9-c5dfa3a54308 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:06.082784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dcaeec-8c8f-4cf5-b4d9-c5dfa3a54308 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:06.082784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 140/268] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:05 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:06.094517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:06.183231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:06.244978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b838db7-253e-453a-99ae-cb783d753234 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:06.280327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:06.280997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.281281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.281560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.282405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] RBAC: Adding query filter params (role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=825464a649f24070aa55b6b968e92f5c) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:06.282716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:06.284201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e41cc9-7f02-4b1b-89d7-039679150c8c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:06.291626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 129/269] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:06 2022] GET /identity/v3/role_assignments?user.id=825464a649f24070aa55b6b968e92f5c&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:06.343955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6e41-bf22-4789-95e1-d3d662bc5776 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.344572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6e41-bf22-4789-95e1-d3d662bc5776 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.345061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6e41-bf22-4789-95e1-d3d662bc5776 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.347292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 141/270] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:06.353942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21391b33-e9f7-42a7-a449-b1ca2e083ac6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.354583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21391b33-e9f7-42a7-a449-b1ca2e083ac6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.355087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21391b33-e9f7-42a7-a449-b1ca2e083ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.373864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-21391b33-e9f7-42a7-a449-b1ca2e083ac6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:06.375453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-21391b33-e9f7-42a7-a449-b1ca2e083ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:06.459201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 130/271] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:06.479206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6e41-bf22-4789-95e1-d3d662bc5776 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:06.515926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e7b40-845a-41d4-acfe-49e40b972d95 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.516441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e7b40-845a-41d4-acfe-49e40b972d95 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.516890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e7b40-845a-41d4-acfe-49e40b972d95 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.518757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 131/272] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:06.525606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.526133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.526607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.551213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:06.552890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:06.586210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:06.596370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6e41-bf22-4789-95e1-d3d662bc5776 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:06.638058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 132/273] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:06.651235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:06.652152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.652497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.652838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.665592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:06.673531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:06.674889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:06.675732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:06.676364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:06.679105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 142/274] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:06.687030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:06.712228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:06.810783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6304dac-c518-4ebe-89c0-7132146c181a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:06.818868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70524ec-de72-473b-80ec-a5e9d66472d6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:06.852874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:06.853435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.853719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.853997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.854751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:06.855088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:06.855982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6436621-87a9-41ef-84c2-b8c6c7556ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:06.856486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681ea89-1f0e-47d9-af7b-760236575302 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:06.856781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436621-87a9-41ef-84c2-b8c6c7556ec3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.857065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436621-87a9-41ef-84c2-b8c6c7556ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.857340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6436621-87a9-41ef-84c2-b8c6c7556ec3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.859028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6436621-87a9-41ef-84c2-b8c6c7556ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:06.860766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6436621-87a9-41ef-84c2-b8c6c7556ec3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:06.863163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 133/275] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:06.864364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c656c59-93e8-497f-a190-f3d6802f23ee None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.864653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c656c59-93e8-497f-a190-f3d6802f23ee None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.864926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c656c59-93e8-497f-a190-f3d6802f23ee None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.870875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 143/277] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:06.871083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 134/277] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:06.871880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c656c59-93e8-497f-a190-f3d6802f23ee None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:06.872566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:06.872857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:06.873132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:06.891344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:06.892533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:06.909796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:06.965479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 144/278] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:06.966691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:06.982276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:06.985728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c656c59-93e8-497f-a190-f3d6802f23ee None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.028476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.029171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.029455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.029734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.030450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:07.031928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:07.052819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 135/279] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:06 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:07.053766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.117330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52831dc7-6b99-4c7c-9388-355fc96f0acd None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.156971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.158016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.158403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.159138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] PATH_INFO: `/v3/users/16caa019d3774f0a8a900cf8a5b000ec` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.174413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] RBAC: Authorizing `identity:get_user(user_id=16caa019d3774f0a8a900cf8a5b000ec)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:07.176070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:07.177675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:07.178515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 145/280] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:06 2022] GET /identity/v3/users/16caa019d3774f0a8a900cf8a5b000ec => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:07.185099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.191584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614d327-2fd7-4d8d-8656-a60da8499535 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.243952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.245118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.245974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.246398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.261420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:07.265088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:07.272459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:07.273398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:07.274794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 136/281] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:07.282573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.309872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d467a-8391-4e1b-b81d-a5f83606c82f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.356668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.357338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.357621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.357896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.358744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, user.id=16caa019d3774f0a8a900cf8a5b000ec, scope.system=all) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:07.359055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:07.360800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:07.366885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 146/282] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Jul 20 21:39:07 2022] GET /identity/v3/role_assignments?user.id=16caa019d3774f0a8a900cf8a5b000ec&scope.system=all&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:07.422079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27da88-5f9b-48ca-a563-f7cee908cc6a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.469518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.470421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.470530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.471132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.474192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:07.474192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:07.474192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bcc17e-043f-41d8-b15d-f45470a959d9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:07.484316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 137/283] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:07.490657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.597080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298f20d-ddd2-4e56-8dfd-9134c713ab22 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.597450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298f20d-ddd2-4e56-8dfd-9134c713ab22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.597651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298f20d-ddd2-4e56-8dfd-9134c713ab22 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.599345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 138/284] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:39:07.604770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.605059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.605342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.619274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6f52-dc0c-4a12-a09d-b218d6241737 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.640148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:07.641686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:07.655467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:07.664346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:07.673118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.673942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.674231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.674513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] PATH_INFO: `/v3/users/7afde253a05444809e8b1b1ce48ecc64` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.677088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] RBAC: Authorizing `identity:get_user(user_id=7afde253a05444809e8b1b1ce48ecc64)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:07.678578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:07.681654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 147/285] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:07 2022] GET /identity/v3/users/7afde253a05444809e8b1b1ce48ecc64 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:07.687153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.801383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 139/286] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:07.827332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d110ef3-527f-42de-a9b9-73c9a2bd7ffb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:07.840163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.870524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:07.871437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:07.871717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:07.871993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:07.873363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:07.874821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:07.879221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 148/287] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:07 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:07.888023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:07.999859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:08.013881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6f92f-1a1c-4a2e-a699-448262623990 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:08.036471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426736f-0da5-4447-a5bd-c3da829ebf93 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:08.050127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:08.051026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:08.051416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:08.051861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:08.069752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:08.078910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:08.079788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:08.080577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 140/288] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:08.089707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:08.092416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:08.093329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:08.093612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:08.093892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88/users/7afde253a05444809e8b1b1ce48ecc64/roles/8ceef45c9f3f430783804f3dfe9ad674` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:08.095104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:08.101486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] RBAC: Authorizing `identity:create_grant(project_id=94662f66ac214075b6277e602fdb8c88, user_id=7afde253a05444809e8b1b1ce48ecc64, role_id=8ceef45c9f3f430783804f3dfe9ad674)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:08.106470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:08.130803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 149/289] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:07 2022] PUT /identity/v3/projects/94662f66ac214075b6277e602fdb8c88/users/7afde253a05444809e8b1b1ce48ecc64/roles/8ceef45c9f3f430783804f3dfe9ad674 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:08.218071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e15d6-c041-4bb1-9eee-6e43d6b0b9e5 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:08.257034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:08.257769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:08.257962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:08.258281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:08.259121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:08.259443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:08.260872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:08.270513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 141/290] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:08.276329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:08.403931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb4b3-490e-4887-b251-dc248f22a89c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:08.445349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:08.453822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:08.454700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:08.455021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:08.455334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] PATH_INFO: `/v3/users/76bcc38a552742c2b8a41c19546ec56e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:08.460662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] RBAC: Authorizing `identity:get_user(user_id=76bcc38a552742c2b8a41c19546ec56e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:08.462288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:08.464226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 150/291] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:08 2022] GET /identity/v3/users/76bcc38a552742c2b8a41c19546ec56e => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:08.469854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:08.540137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:08.580738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73542-31ec-4833-b07d-865b24a3ec0e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:08.617278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:08.617960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:08.618247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:08.618531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:08.693692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:08.695274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:08.766856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:08.809125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 142/292] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:08.814633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:08.943516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce2dfb-4657-4aa0-a2fc-3a956ab4f3fd None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:08.989627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab72823-7710-438d-a30b-0de414a70feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:08.990834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab72823-7710-438d-a30b-0de414a70feb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:08.991369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab72823-7710-438d-a30b-0de414a70feb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:08.991847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab72823-7710-438d-a30b-0de414a70feb None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:08.993857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab72823-7710-438d-a30b-0de414a70feb None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:08.995558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab72823-7710-438d-a30b-0de414a70feb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:08.997472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 151/293] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:08 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:09.003572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:09.158783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66340d-f116-429a-ace7-6a92a84188fa None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:09.208193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:09.209226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:09.209708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:09.210190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:09.211256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=76bcc38a552742c2b8a41c19546ec56e) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:09.212029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:09.213688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d8f96-aff2-4eeb-ac30-9a0f9dd3115c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:09.223960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 143/294] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:09 2022] GET /identity/v3/role_assignments?user.id=76bcc38a552742c2b8a41c19546ec56e&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:09.752251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99b4c3-cffa-4596-a43c-5caa35a9a234 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:09.752955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99b4c3-cffa-4596-a43c-5caa35a9a234 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:09.753484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99b4c3-cffa-4596-a43c-5caa35a9a234 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:09.755199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 152/295] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:09.761522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c837-ce18-47d4-9a69-c889a241c893 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:09.762026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c837-ce18-47d4-9a69-c889a241c893 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:09.762161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421c837-ce18-47d4-9a69-c889a241c893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:09.784236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c421c837-ce18-47d4-9a69-c889a241c893 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:09.785728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c421c837-ce18-47d4-9a69-c889a241c893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:09.821943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:09.872047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 144/296] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:09.893835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99b4c3-cffa-4596-a43c-5caa35a9a234 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:09.926021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e7104-6bfa-4645-8e39-5bd4f20e79a2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:09.926656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e7104-6bfa-4645-8e39-5bd4f20e79a2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:09.927186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e7104-6bfa-4645-8e39-5bd4f20e79a2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:09.929042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 145/297] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:09.934717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6066-7bb2-4a71-884c-6010535f05b8 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:09.935272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6066-7bb2-4a71-884c-6010535f05b8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:09.935779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b6066-7bb2-4a71-884c-6010535f05b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:09.954103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-820b6066-7bb2-4a71-884c-6010535f05b8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:09.956315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-820b6066-7bb2-4a71-884c-6010535f05b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:09.962605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.007342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99b4c3-cffa-4596-a43c-5caa35a9a234 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.033110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 146/298] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:10.035284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd503b7-23b5-4e06-bcf6-c20e4f3647c0 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.035284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd503b7-23b5-4e06-bcf6-c20e4f3647c0 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.035284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd503b7-23b5-4e06-bcf6-c20e4f3647c0 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.038537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 147/299] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:10.044387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.044695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.045155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.054783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.056331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.056705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.057208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.065333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:10.066547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:10.070292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:10.080681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:10.081687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 153/300] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:10.088238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.155113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 148/301] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:10.156569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.180878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.195213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c2308-d283-4b28-8946-c5d31f1fa2a9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.220168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.232992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.233789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.234072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.234347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.244965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:10.252778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:10.253691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:10.254496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 154/302] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:10.255190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.279032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12662eb9-40a9-4cf7-8c34-d7cdea09bd5b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.314736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.315443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.315731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.316013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.316773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:10.317085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.318460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.327239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 149/303] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:10.328045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.369730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4931f-1f52-4285-acfb-185a5241b925 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.408982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.409782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.410068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.410344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.411547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.413415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.420731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 155/304] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:10.420731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.420958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.440722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4935183-0432-4aa0-bc9d-185da2122e96 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.475365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.476269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.476546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.476814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.478314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:10.478836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.479300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.480318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.489130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 150/305] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:10.489130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.534405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e65727-d385-4b8c-8b41-991d2f721c62 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.569193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.570064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.570341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.570612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] PATH_INFO: `/v3/users/16caa019d3774f0a8a900cf8a5b000ec` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.572359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] RBAC: Authorizing `identity:get_user(user_id=16caa019d3774f0a8a900cf8a5b000ec)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.574164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.580959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 156/306] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/users/16caa019d3774f0a8a900cf8a5b000ec => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:10.580959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.604757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9facec63-d6af-4b54-9072-9c7030619a7a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.643213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.643889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.644173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.644453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] PATH_INFO: `/v3/groups` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.645147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.646683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.665749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 151/307] 10.222.0.86 () {60 vars in 1134 bytes} [Wed Jul 20 21:39:10 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:10.666855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.675941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.691331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568c5b-d82c-47b1-86e9-b132b5133de8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.714580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.734380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd1174-74c1-4197-ac60-2f8483fee508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.736671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd1174-74c1-4197-ac60-2f8483fee508 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.737221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd1174-74c1-4197-ac60-2f8483fee508 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.737679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd1174-74c1-4197-ac60-2f8483fee508 None admin] PATH_INFO: `/v3/users/6904890b25fa4c1995b8d64a8c6c548e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.756520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd1174-74c1-4197-ac60-2f8483fee508 None admin] RBAC: Authorizing `identity:get_user(user_id=6904890b25fa4c1995b8d64a8c6c548e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.758121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd1174-74c1-4197-ac60-2f8483fee508 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.765019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 157/308] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/users/6904890b25fa4c1995b8d64a8c6c548e => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:10.765019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0d32-bc97-4e97-8b74-98a002d21b33 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.765019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0d32-bc97-4e97-8b74-98a002d21b33 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.765019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0d32-bc97-4e97-8b74-98a002d21b33 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.765019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 158/309] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:10.768682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0d32-bc97-4e97-8b74-98a002d21b33 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:10.808079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745f9ab-7ebd-4797-8ef0-d4c44449d460 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.848484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f82bd4-cdf3-4ae9-9553-47005f498e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.849363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f82bd4-cdf3-4ae9-9553-47005f498e1b None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.849649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f82bd4-cdf3-4ae9-9553-47005f498e1b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.849929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f82bd4-cdf3-4ae9-9553-47005f498e1b None admin] PATH_INFO: `/v3/system/users/16caa019d3774f0a8a900cf8a5b000ec/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.851509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f82bd4-cdf3-4ae9-9553-47005f498e1b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=16caa019d3774f0a8a900cf8a5b000ec, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.854159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f82bd4-cdf3-4ae9-9553-47005f498e1b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.865072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 152/310] 10.222.0.86 () {58 vars in 1334 bytes} [Wed Jul 20 21:39:10 2022] PUT /identity/v3/system/users/16caa019d3774f0a8a900cf8a5b000ec/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:10.866374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae925-ac61-4537-865b-be5260502f6f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.866374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae925-ac61-4537-865b-be5260502f6f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.866601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bae925-ac61-4537-865b-be5260502f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.884969 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-50bae925-ac61-4537-865b-be5260502f6f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:10.886215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-50bae925-ac61-4537-865b-be5260502f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:10.887262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0d32-bc97-4e97-8b74-98a002d21b33 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:10.935891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:10.936705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:10.936893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:10.937176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:10.938039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] RBAC: Adding query filter params (role.id=8ceef45c9f3f430783804f3dfe9ad674, user.id=6904890b25fa4c1995b8d64a8c6c548e, scope.system=all) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:10.938356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:10.941261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:10.948524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 159/311] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Jul 20 21:39:10 2022] GET /identity/v3/role_assignments?user.id=6904890b25fa4c1995b8d64a8c6c548e&scope.system=all&role.id=8ceef45c9f3f430783804f3dfe9ad674 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:10.979492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:10.996078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 153/312] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:11.013327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:11.064711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:11.130726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd8e11-3d3a-4ef4-b694-04432a12e61c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:11.167150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:11.168303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:11.168779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:11.169270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:11.180815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:11.189034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:11.189947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:11.190747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:11.193507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 160/313] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:11.199795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bae925-ac61-4537-865b-be5260502f6f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:11.306978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bae925-ac61-4537-865b-be5260502f6f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:11.342291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:11.342883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:11.343195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:11.343471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:11.344234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:11.344542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:11.345922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:11.360987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 154/314] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:11.364144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:11.468175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48b297e-1a5c-4017-9871-29c1ea4582a8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:11.469732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:11.504819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:11.505694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:11.505977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:11.506257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] PATH_INFO: `/v3/users/7afde253a05444809e8b1b1ce48ecc64` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:11.507892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=7afde253a05444809e8b1b1ce48ecc64)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:11.509503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:11.511487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 161/315] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:11 2022] GET /identity/v3/users/7afde253a05444809e8b1b1ce48ecc64 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:11.516839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:11.580633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:11.626928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7755db-fd40-4eb2-b4c5-c3a84c73da2f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:11.662294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:11.662952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:11.663267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:11.663547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:11.739148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:11.740685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:11.799912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:11.848735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 155/316] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:11.850072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:11.959843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d83f0-ebb5-4127-ad44-3f4a6218c2d4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:11.999134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f763902-4d9a-4370-8485-60e8355970cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:12.000347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763902-4d9a-4370-8485-60e8355970cf None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:12.000819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763902-4d9a-4370-8485-60e8355970cf None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:12.001303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f763902-4d9a-4370-8485-60e8355970cf None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:12.002681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763902-4d9a-4370-8485-60e8355970cf None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:12.004399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f763902-4d9a-4370-8485-60e8355970cf None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:12.006589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 162/317] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:11 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:12.013035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:12.119394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aa70f-b33d-487e-a63e-7996523cf249 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:12.155390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:12.156051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:12.156333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:12.156608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:12.157442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] RBAC: Adding query filter params (role.id=8ceef45c9f3f430783804f3dfe9ad674, scope.project.id=94662f66ac214075b6277e602fdb8c88, user.id=7afde253a05444809e8b1b1ce48ecc64) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:12.157755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:12.159232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f06dc4-52fe-4c9f-a0b7-9123fe3dd55f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:12.171233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 156/318] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:12 2022] GET /identity/v3/role_assignments?user.id=7afde253a05444809e8b1b1ce48ecc64&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=8ceef45c9f3f430783804f3dfe9ad674 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:12.836828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6259f8-99d7-47c0-9d2d-5a527aa3a0a0 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:12.837193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6259f8-99d7-47c0-9d2d-5a527aa3a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:12.837661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6259f8-99d7-47c0-9d2d-5a527aa3a0a0 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:12.839232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 163/319] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:12.844217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78630-2f5d-4bd5-911f-ae4824f195b6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:12.844515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78630-2f5d-4bd5-911f-ae4824f195b6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:12.844795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa78630-2f5d-4bd5-911f-ae4824f195b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:12.862684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-caa78630-2f5d-4bd5-911f-ae4824f195b6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:12.863920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-caa78630-2f5d-4bd5-911f-ae4824f195b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:12.881855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:12.934888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 157/320] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:12.957371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6259f8-99d7-47c0-9d2d-5a527aa3a0a0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:12.982636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21a7c7-2283-4b33-b149-590124f76c86 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:12.982923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21a7c7-2283-4b33-b149-590124f76c86 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:12.983241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21a7c7-2283-4b33-b149-590124f76c86 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:12.984874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 158/321] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:12.989917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:12.990199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:12.990482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.005030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.008156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:13.009963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:13.072729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6259f8-99d7-47c0-9d2d-5a527aa3a0a0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.086302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 159/322] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:13.108347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.112620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed687e70-80fd-4752-89cc-ac90bbbc6072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.113417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed687e70-80fd-4752-89cc-ac90bbbc6072 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.113699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed687e70-80fd-4752-89cc-ac90bbbc6072 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.113973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed687e70-80fd-4752-89cc-ac90bbbc6072 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.115162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed687e70-80fd-4752-89cc-ac90bbbc6072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.116612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed687e70-80fd-4752-89cc-ac90bbbc6072 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.118448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 164/323] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:13.118448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c17ea-bd79-44e8-9cf6-81739ba56998 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.118448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c17ea-bd79-44e8-9cf6-81739ba56998 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.118448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c17ea-bd79-44e8-9cf6-81739ba56998 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.118448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 165/324] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:13.123788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c17ea-bd79-44e8-9cf6-81739ba56998 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.191903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.209932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc44a9-6635-48d4-a99f-8ef2e2a9e14a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.226216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c17ea-bd79-44e8-9cf6-81739ba56998 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.233505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.244479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be94b29-5f1f-49f3-b072-199a7da2d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.245322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be94b29-5f1f-49f3-b072-199a7da2d966 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.245620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be94b29-5f1f-49f3-b072-199a7da2d966 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.245901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be94b29-5f1f-49f3-b072-199a7da2d966 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.256219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be94b29-5f1f-49f3-b072-199a7da2d966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:13.261161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.261814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.262091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.262358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] PATH_INFO: `/v3/groups` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.263040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-9be94b29-5f1f-49f3-b072-199a7da2d966 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:13.263991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:13.264992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:13.265814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:13.266440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.266509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 160/325] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:13.268272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.268551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.268822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.295095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:13.295301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 166/326] 10.222.0.86 () {60 vars in 1134 bytes} [Wed Jul 20 21:39:13 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:13.295301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.295474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:13.370697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 161/327] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:13.392996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.406755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f1d6-d3f6-4244-9aec-909dd3239916 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.446970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.447677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.447962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.448235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.448991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:13.449297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.450668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.460471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 167/328] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:13.466150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.481507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.504748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.525032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416cbf2-1c3d-4567-9205-f45a020207db None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.570014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.571715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.572142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.572563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.586149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:13.594536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:13.595550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:13.596402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 162/329] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:13.603996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34695af-ea47-498c-ae66-f9527c94f01f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.604476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.656414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.657229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.657507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.657779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] PATH_INFO: `/v3/users/16caa019d3774f0a8a900cf8a5b000ec` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.659120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] RBAC: Authorizing `identity:get_user(user_id=16caa019d3774f0a8a900cf8a5b000ec)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.660613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.662608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 168/330] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/users/16caa019d3774f0a8a900cf8a5b000ec => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:13.668418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.746404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf2cfe-ac30-4097-9d4a-70cffc4e4a32 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.782211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.787558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:13.790850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494d327-73da-46e9-b6d1-6e39d2557b30 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.794602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.796314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.796675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.797024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.797851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:13.798234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.799731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.817424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 163/331] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:13.819978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:13.840429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.841129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.841412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.841686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.842827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] RBAC: Adding query filter params (role.id=daae1e7e6e714a95b85fff4ea4b37745, user.id=16caa019d3774f0a8a900cf8a5b000ec, scope.system=all) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:13.843173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.844918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.867145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 169/332] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/role_assignments?user.id=16caa019d3774f0a8a900cf8a5b000ec&scope.system=all&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:13.947572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15b8c1-f6cb-4832-90ec-8d34e69537fe None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:13.986410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2a455-c4fd-4609-af33-f98fc09a60f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:13.987337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a455-c4fd-4609-af33-f98fc09a60f7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:13.987618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a455-c4fd-4609-af33-f98fc09a60f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:13.987892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2a455-c4fd-4609-af33-f98fc09a60f7 None admin] PATH_INFO: `/v3/users/6904890b25fa4c1995b8d64a8c6c548e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:13.989141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a455-c4fd-4609-af33-f98fc09a60f7 None admin] RBAC: Authorizing `identity:get_user(user_id=6904890b25fa4c1995b8d64a8c6c548e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:13.990595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2a455-c4fd-4609-af33-f98fc09a60f7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:13.993245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 164/333] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:13 2022] GET /identity/v3/users/6904890b25fa4c1995b8d64a8c6c548e => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:13.999180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:14.137398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331b434-7884-4441-b328-0a24b2660bdd None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:14.178052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46593687-c674-439d-bd7e-70385c667174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:14.179234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593687-c674-439d-bd7e-70385c667174 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:14.179660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593687-c674-439d-bd7e-70385c667174 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:14.180121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593687-c674-439d-bd7e-70385c667174 None admin] PATH_INFO: `/v3/system/users/6904890b25fa4c1995b8d64a8c6c548e/roles/8ceef45c9f3f430783804f3dfe9ad674` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:14.181851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46593687-c674-439d-bd7e-70385c667174 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6904890b25fa4c1995b8d64a8c6c548e, role_id=8ceef45c9f3f430783804f3dfe9ad674)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:14.183432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46593687-c674-439d-bd7e-70385c667174 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:14.198289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 170/334] 10.222.0.86 () {58 vars in 1334 bytes} [Wed Jul 20 21:39:13 2022] PUT /identity/v3/system/users/6904890b25fa4c1995b8d64a8c6c548e/roles/8ceef45c9f3f430783804f3dfe9ad674 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:15.756029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140aad80-3b13-49ed-bb22-3bbafc2a74c6 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:15.757002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140aad80-3b13-49ed-bb22-3bbafc2a74c6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:15.757579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140aad80-3b13-49ed-bb22-3bbafc2a74c6 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:15.760000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 165/335] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:15.769860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:15.770736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:15.771256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:15.800104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:15.801680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:15.818235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:15.890731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 171/336] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:15.935543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140aad80-3b13-49ed-bb22-3bbafc2a74c6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:16.017095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:16.066378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140aad80-3b13-49ed-bb22-3bbafc2a74c6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:16.113708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6860c-c04b-4253-8731-c0b450f56211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:16.114509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6860c-c04b-4253-8731-c0b450f56211 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.114793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6860c-c04b-4253-8731-c0b450f56211 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.115088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6860c-c04b-4253-8731-c0b450f56211 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.130399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6860c-c04b-4253-8731-c0b450f56211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-aea6860c-c04b-4253-8731-c0b450f56211 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:16.139458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:16.140357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:16.141192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:16.145517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 166/337] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:16.146916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:16.270975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07155c9d-cece-44c3-90d4-c5a1d836cee2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:16.272394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbec9-c19f-48db-a69e-fe52ac9fb2ef None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.272677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbec9-c19f-48db-a69e-fe52ac9fb2ef None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.272951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9dbec9-c19f-48db-a69e-fe52ac9fb2ef None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.274439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 167/338] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:39:16.288712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.289833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.290531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.320409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:16.322156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:16.329232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:16.329920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.330199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.330467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.331247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:16.331551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:16.332949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:16.350929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 172/339] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:16.357504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:16.377541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:16.473670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 168/340] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:16.522878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:16.526192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07d57-3421-4980-9dc0-758bae59620a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:16.586023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faba2b-d8d7-4b43-aced-c5150d722a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:16.587346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faba2b-d8d7-4b43-aced-c5150d722a1f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.587637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faba2b-d8d7-4b43-aced-c5150d722a1f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.587903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faba2b-d8d7-4b43-aced-c5150d722a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.654137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:16.687380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faba2b-d8d7-4b43-aced-c5150d722a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:16.689187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faba2b-d8d7-4b43-aced-c5150d722a1f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:16.704099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:16.724326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63cfac-aa43-4281-9e43-3a43694ccc57 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:16.782627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e4995-2128-4176-89a9-c99ff81ed9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:16.784116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e4995-2128-4176-89a9-c99ff81ed9e6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.784541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e4995-2128-4176-89a9-c99ff81ed9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.784973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e4995-2128-4176-89a9-c99ff81ed9e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.804546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e4995-2128-4176-89a9-c99ff81ed9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:16.809324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 173/341] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:39:16.811255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2c56-d035-431e-be0c-65d9c68d1789 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.811752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2c56-d035-431e-be0c-65d9c68d1789 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.812229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2c56-d035-431e-be0c-65d9c68d1789 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.813740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 174/342] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:16.814653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf2c56-d035-431e-be0c-65d9c68d1789 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-048e4995-2128-4176-89a9-c99ff81ed9e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:16.817509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:16.818576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:16.819619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:16.822745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 169/343] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:16.824146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:16.824593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:16.825009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:16.850957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:16.852445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:16.948625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 170/344] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:16.949513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:16.998855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf2c56-d035-431e-be0c-65d9c68d1789 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.017311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.048682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.064146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.065045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.065338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.065616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.067636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.069096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.071816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 175/345] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:16 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:17.072609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.138586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e90e8e-901d-435e-bd27-c128ed5d96ac None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.197426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.198446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.198947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.199438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.200386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:17.200799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8da35-6380-4bfc-a93c-8950ecf77b1a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.201002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.202538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.215825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 171/346] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:17.216960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.239523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.240561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.241017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.241539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.254203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:17.263576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:17.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:17.265455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:17.268365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 176/347] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:17.269432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.314868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.341402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27494778-525c-4e47-be5d-7917c83496a0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.344435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.384268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da9808-8602-41dd-92d6-614c6775c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.385530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da9808-8602-41dd-92d6-614c6775c073 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.386091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da9808-8602-41dd-92d6-614c6775c073 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.386546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da9808-8602-41dd-92d6-614c6775c073 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.389191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0fe00-7d12-4370-ae1f-5ab918adcf4a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.393841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da9808-8602-41dd-92d6-614c6775c073 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.395574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da9808-8602-41dd-92d6-614c6775c073 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.397899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 172/348] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:17.400016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b98108-3157-4994-ad17-9b811576baa6 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.400508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b98108-3157-4994-ad17-9b811576baa6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.400963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b98108-3157-4994-ad17-9b811576baa6 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.402525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 173/349] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:17.403425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b98108-3157-4994-ad17-9b811576baa6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.432614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfcea5-36ed-4dee-a36c-34a11809af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.433559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfcea5-36ed-4dee-a36c-34a11809af2c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.433998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfcea5-36ed-4dee-a36c-34a11809af2c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.434406 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfcea5-36ed-4dee-a36c-34a11809af2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.509145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfcea5-36ed-4dee-a36c-34a11809af2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.510869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfcea5-36ed-4dee-a36c-34a11809af2c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.519849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b98108-3157-4994-ad17-9b811576baa6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.552990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.561300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.562355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.562843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.563381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.564447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:17.565113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.566694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.577793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 174/350] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:17.579005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.621319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.623210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 177/351] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:17.625408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.625969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.626442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.646540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:17.648228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:17.722839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbfea-916d-45c8-bc8b-8e082a0d39ac None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.730540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 178/352] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:17.731687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.771278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.772307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.772788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.773258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.774903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:17.775472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.777155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.783903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 175/353] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:17.785461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:17.816913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.885218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29868c89-d3e9-4989-8dd9-e5a0c58b3531 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.909172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:17.920498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf42a1-70f2-426a-a345-8fa73de2fc43 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:17.938617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794835d2-021d-47b4-9195-a87d00692b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.939724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794835d2-021d-47b4-9195-a87d00692b11 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.939724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794835d2-021d-47b4-9195-a87d00692b11 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.939983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794835d2-021d-47b4-9195-a87d00692b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.963282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:17.964402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:17.964823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:17.965309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:17.966932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:17.968618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:17.970608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 176/354] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:17.971618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.030423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794835d2-021d-47b4-9195-a87d00692b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.032108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794835d2-021d-47b4-9195-a87d00692b11 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.089885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f21b4-1278-49fa-9575-6b36191aef6d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.124953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:18.136804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b34fb8-57ee-4ce1-892c-9d5ce05ea2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.137621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34fb8-57ee-4ce1-892c-9d5ce05ea2f3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.137906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34fb8-57ee-4ce1-892c-9d5ce05ea2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.138180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b34fb8-57ee-4ce1-892c-9d5ce05ea2f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.152353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b34fb8-57ee-4ce1-892c-9d5ce05ea2f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-d5b34fb8-57ee-4ce1-892c-9d5ce05ea2f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:18.160211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:18.161145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 21:39:18.161978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:18.167210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 177/355] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:18.168618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648535e4-1f69-446b-8feb-53221bddf7d2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.168909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648535e4-1f69-446b-8feb-53221bddf7d2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.169189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648535e4-1f69-446b-8feb-53221bddf7d2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.170697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 178/356] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:18.174744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 179/357] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:18.174890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648535e4-1f69-446b-8feb-53221bddf7d2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.176395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1bf08-57a7-45eb-991c-dac36e9cc3e4 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.176684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1bf08-57a7-45eb-991c-dac36e9cc3e4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.176962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1bf08-57a7-45eb-991c-dac36e9cc3e4 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.178577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 180/358] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:18.180777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1bf08-57a7-45eb-991c-dac36e9cc3e4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.204214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:18.307299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648535e4-1f69-446b-8feb-53221bddf7d2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.327261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1bf08-57a7-45eb-991c-dac36e9cc3e4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.350019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492153a6-8e75-43ed-b402-e9dcc3453a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.351717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492153a6-8e75-43ed-b402-e9dcc3453a76 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.351717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492153a6-8e75-43ed-b402-e9dcc3453a76 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.351717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492153a6-8e75-43ed-b402-e9dcc3453a76 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.353981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492153a6-8e75-43ed-b402-e9dcc3453a76 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.358187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492153a6-8e75-43ed-b402-e9dcc3453a76 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.358187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 179/359] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:18.359475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfffafb-fae3-4e7f-9461-4b079d8dfb22 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.360343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfffafb-fae3-4e7f-9461-4b079d8dfb22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.360343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfffafb-fae3-4e7f-9461-4b079d8dfb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.380501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.381209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.381495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.381703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ddfffafb-fae3-4e7f-9461-4b079d8dfb22 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:18.381803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.382521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:18.382832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.382928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ddfffafb-fae3-4e7f-9461-4b079d8dfb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:18.384978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.394764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 181/360] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:18.395627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.439027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:18.464917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 180/361] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:18.466079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.466372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.466653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.485876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:18.488224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:18.506839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:18.523673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43473c-de54-4bd9-b21b-7f9a3a721060 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.569158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.570039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.570321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.570642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.573365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.574940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.577049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 182/362] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:18.577741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.583442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 181/363] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:18.584297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.700203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f444d-12b4-4bd1-b8ce-6bd8f59df3b1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.705094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:18.705429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a5f2b-c76c-445c-adca-ea9efb747069 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.745912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.746796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.747103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.747354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] PATH_INFO: `/v3/users/6904890b25fa4c1995b8d64a8c6c548e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.748552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] RBAC: Authorizing `identity:get_user(user_id=6904890b25fa4c1995b8d64a8c6c548e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.749997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.751501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.752000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 182/364] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/users/6904890b25fa4c1995b8d64a8c6c548e => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:18.752198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.752445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.752722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.752822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.753683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:18.754000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.755971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.765461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 183/365] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:18.766654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.787721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:18.896970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c867e4-0ac1-48f0-9b63-3e2bc3c71814 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.906262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2b84d-a7ba-4bef-ada3-39e165afa5b7 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:18.947703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.948530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.948826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.949110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.949429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:18.950537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:18.950770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:18.951048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:18.953997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:18.954774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:18.956959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 184/366] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:18.957751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:18.960478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:18.970269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:18.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:18.972043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:18.973177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 183/367] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:18.973338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:19.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:19.088857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:19.115576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340892b-4ded-4b55-b544-e529ac0fd384 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.116331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76330d-6851-4f35-9438-24f1d89535be None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.153982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.154972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.155377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.155771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.156001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.156813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] RBAC: Adding query filter params (role.id=8ceef45c9f3f430783804f3dfe9ad674, user.id=6904890b25fa4c1995b8d64a8c6c548e, scope.system=all) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:19.157117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.157314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:19.157672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.158080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.159255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:19.170153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:19.174211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 184/368] 10.222.0.86 () {56 vars in 1308 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/role_assignments?user.id=6904890b25fa4c1995b8d64a8c6c548e&scope.system=all&role.id=8ceef45c9f3f430783804f3dfe9ad674 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:19.174211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:19.180178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:19.181335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:19.182123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:19.189792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 185/369] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:19.191213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:19.293545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9c8fa-280f-4355-a62e-466dc82bd376 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.331423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.332119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.332403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.332681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.333527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=94662f66ac214075b6277e602fdb8c88) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:19.333845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:19.335332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:19.339373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:19.339536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:19.342848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 185/370] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:19 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:19.343742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:19.355157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c06699-2a48-434e-bf54-8e3f0aaca022 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.433755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.435030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.435540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.436015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.437078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:19.437598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:19.439365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:19.459922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 186/371] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:19.466735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:19.469194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08b60a-3d31-4435-aae8-7b02efbb283e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.541268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.541956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.542236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.542510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.543305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:19.543669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:19.545020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:19.565232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 186/372] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:19.572176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:19.685487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6d54-76a7-43d3-b9d0-d70c97655668 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.697881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:19.757973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecefe4-49bc-47f3-b7b7-02b119b0da2b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:19.770862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.771555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.771840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.772113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.810457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:19.811149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:19.811329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:19.811609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:19.811885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:19.878937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:19.880785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:19.907839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:19.909426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:19.979337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.026793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 187/373] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:20.031528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.034192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 187/374] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:39:20.040380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.146460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebae89c-26bb-413f-ae7d-b2e57feebf3e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.186406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c45b4-89bd-4815-95a9-b8dad2a7895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.187436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c45b4-89bd-4815-95a9-b8dad2a7895c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.188237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c45b4-89bd-4815-95a9-b8dad2a7895c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.188740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c45b4-89bd-4815-95a9-b8dad2a7895c None admin] PATH_INFO: `/v3/projects/feb77f81bd3a4077b594b0faf70919df` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.190247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c45b4-89bd-4815-95a9-b8dad2a7895c None admin] RBAC: Authorizing `identity:get_project(project_id=feb77f81bd3a4077b594b0faf70919df)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:20.190959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.192239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c45b4-89bd-4815-95a9-b8dad2a7895c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:20.194242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 188/375] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/projects/feb77f81bd3a4077b594b0faf70919df => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:20.196513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611b857-84ac-4a60-a00b-2dca805ff865 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.196513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611b857-84ac-4a60-a00b-2dca805ff865 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.196513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611b857-84ac-4a60-a00b-2dca805ff865 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.197694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 189/376] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:20.199987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611b857-84ac-4a60-a00b-2dca805ff865 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.241509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.276828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d608754-88b3-4540-818d-7b73039f45a5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.333400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b635251-e836-462b-942a-eb3a5f6c5ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.334454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635251-e836-462b-942a-eb3a5f6c5ee2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.335187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635251-e836-462b-942a-eb3a5f6c5ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.335476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b635251-e836-462b-942a-eb3a5f6c5ee2 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.338088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635251-e836-462b-942a-eb3a5f6c5ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:20.340309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b635251-e836-462b-942a-eb3a5f6c5ee2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:20.342537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 188/377] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:20.343396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611b857-84ac-4a60-a00b-2dca805ff865 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.349014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.351453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.352479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.373725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:20.375998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:20.395564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.397289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.397844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.398279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] PATH_INFO: `/v3/groups/b5d356c39aff40db8ecbb7a5be593e94` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.405897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] RBAC: Authorizing `identity:get_group(group_id=b5d356c39aff40db8ecbb7a5be593e94)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:20.408253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:20.415368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 190/378] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/groups/b5d356c39aff40db8ecbb7a5be593e94 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:20.416210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.470427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 189/379] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:20.472125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.533006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.545331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.556806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70449d48-f1bb-4c03-995d-9617ed022622 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.598094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.599337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.599824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.600292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.600850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5171d-c84b-458f-b2b8-30329b93c5b9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.602537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:20.604238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:20.606203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 191/380] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:20.607250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.649509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.650977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.651558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.652029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.653070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] RBAC: Adding query filter params (group.id=b5d356c39aff40db8ecbb7a5be593e94, role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=feb77f81bd3a4077b594b0faf70919df) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:20.654509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:20.656208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:20.662258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 190/381] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/role_assignments?group.id=b5d356c39aff40db8ecbb7a5be593e94&scope.project.id=feb77f81bd3a4077b594b0faf70919df&role.id=62c943ce95d54a1999a447dca6a1103b => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:20.663493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.746088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac39366-e3d9-437e-b02b-9df915c52b56 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.792284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.793113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.793398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.793676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.807914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:20.817069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:20.818478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:20.819995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 192/382] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:20.825091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:20.869316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a5497-1f70-4e2d-9a5f-2e82db3e7437 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:20.880475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.892584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:20.930811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:20.931816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:20.932106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:20.932431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:20.933275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=94662f66ac214075b6277e602fdb8c88) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:20.933586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:20.935039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:20.946987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 191/383] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 260 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:20.993248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7db569-0476-4365-abe3-accee04a9ff2 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:21.035116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:21.036246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.036739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.037233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.038293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:21.038865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:21.040595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:21.051674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 193/384] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:21.058493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:21.194513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489f442-1de9-4659-901f-7b76c3cd833d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:21.229896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:21.230756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.231034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.231334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.232522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:21.233977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:21.236012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 192/385] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:21 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:21.244159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:21.345426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:21.363979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02790b32-0dee-48c6-be1e-3270e27983a3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:21.406318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d8930-c8f7-4968-b95f-92b7e0528006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:21.407212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d8930-c8f7-4968-b95f-92b7e0528006 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.407492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d8930-c8f7-4968-b95f-92b7e0528006 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.407762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d8930-c8f7-4968-b95f-92b7e0528006 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.408956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d8930-c8f7-4968-b95f-92b7e0528006 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:21.410394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d8930-c8f7-4968-b95f-92b7e0528006 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:21.413397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 194/386] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:21 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:21.418732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:21.471869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:21.526113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33fbd-1f64-48c1-8757-eaf85ab02a90 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:21.561069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7acf269-92ae-4c97-9d80-c0f034f96e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:21.561952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acf269-92ae-4c97-9d80-c0f034f96e65 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.562227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acf269-92ae-4c97-9d80-c0f034f96e65 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.562495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7acf269-92ae-4c97-9d80-c0f034f96e65 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/groups/305c4d70bd174764a1585a55545c6b3f/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.564487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acf269-92ae-4c97-9d80-c0f034f96e65 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0857875f64efaa76e5af8b36c50ba, group_id=305c4d70bd174764a1585a55545c6b3f, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:21.565922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7acf269-92ae-4c97-9d80-c0f034f96e65 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:21.583232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 193/387] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Jul 20 21:39:21 2022] PUT /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/groups/305c4d70bd174764a1585a55545c6b3f/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:21.601173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5163-77f5-4fc2-94bf-b289c42c6447 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.601517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5163-77f5-4fc2-94bf-b289c42c6447 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.601825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a5163-77f5-4fc2-94bf-b289c42c6447 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.603837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 195/388] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:21.621994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.622323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.622549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.644223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:21.645433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:21.733475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 194/389] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:21.759558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a5163-77f5-4fc2-94bf-b289c42c6447 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:21.886189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a5163-77f5-4fc2-94bf-b289c42c6447 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:21.923683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:21.924760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:21.925187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:21.925611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:21.937159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:21.945735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:21.946831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:21.947856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:21.950407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 196/390] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:21.956663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:21.998526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:22.071722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94f665-fee8-4333-88be-d4c2ddc5e0cd None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:22.116781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:22.117494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.117779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.118051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.119001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:22.119340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:22.120765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee3b07-68c4-45b7-aa5d-ae2f27984ada None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:22.134800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 195/391] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:22.140393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:22.161856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:22.256048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaebba-adab-4cba-b4c5-2f48777ab59c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:22.265419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211bc3-18c4-4723-a831-0d6740edda21 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.265751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211bc3-18c4-4723-a831-0d6740edda21 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.265974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06211bc3-18c4-4723-a831-0d6740edda21 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.267622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 196/392] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:22.282367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.282690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.282924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.299727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:22.300646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.300924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.301199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.302432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:22.303927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:22.305939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 197/393] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:22 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:22.309666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:22.310940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:22.311678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:22.399210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:22.421688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453c1b1-8bde-4e3b-bc58-f1219cf3f105 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:22.421984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 197/394] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:22.456160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:22.460027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:22.460908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.461725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.462002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.463293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:22.464764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:22.466897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 198/395] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:22 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:22.472487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:22.511523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:22.650367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50de990-cee1-4d7e-befc-74a25dee5fc6 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:22.661487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e15e-26d8-4ecf-a3eb-a981915a55e9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:22.699994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:22.700933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.701220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.701497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/groups/305c4d70bd174764a1585a55545c6b3f/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7f0857875f64efaa76e5af8b36c50ba, group_id=305c4d70bd174764a1585a55545c6b3f, role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:22.705366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:22.711780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:22.712934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.713273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.713549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.720640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 199/396] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Jul 20 21:39:22 2022] PUT /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba/groups/305c4d70bd174764a1585a55545c6b3f/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:39:22.737721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:22.750299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:22.751371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:22.752193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:22.754442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 198/397] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:22.760712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:22.773501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:22.878349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d31be-ad70-44ea-a3cd-4f5a21611a77 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:22.915551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:22.917096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:22.917438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:22.917717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:22.918538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:22.918854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:22.920273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:22.930452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 200/398] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:22.936088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:23.050645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:23.064578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15165752-4453-428c-a90f-62ef6e113ba4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:23.100557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:23.101430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.101710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.102563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.103783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:23.105252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:23.107204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 199/399] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:22 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:23.113019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:23.190546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:23.236944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02b4a-3c7d-43fb-8c0c-15581ac5f74e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:23.282848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b103d0-5167-41ff-b4f5-417fe297c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:23.283805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b103d0-5167-41ff-b4f5-417fe297c668 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.284190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b103d0-5167-41ff-b4f5-417fe297c668 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.284519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b103d0-5167-41ff-b4f5-417fe297c668 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.285717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b103d0-5167-41ff-b4f5-417fe297c668 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:23.287256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b103d0-5167-41ff-b4f5-417fe297c668 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:23.288969 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 201/400] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:23 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:23.295278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:23.424283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7125870f-7a5c-42f8-8cb4-83555c4de7b8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:23.467273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb233c7f-6cd0-4e91-88b9-927a35152f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:23.468173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb233c7f-6cd0-4e91-88b9-927a35152f0f None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.468458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb233c7f-6cd0-4e91-88b9-927a35152f0f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.468738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb233c7f-6cd0-4e91-88b9-927a35152f0f None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88/groups/305c4d70bd174764a1585a55545c6b3f/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.470840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb233c7f-6cd0-4e91-88b9-927a35152f0f None admin] RBAC: Authorizing `identity:create_grant(project_id=94662f66ac214075b6277e602fdb8c88, group_id=305c4d70bd174764a1585a55545c6b3f, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:23.472348 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb233c7f-6cd0-4e91-88b9-927a35152f0f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:23.491828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 200/401] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Jul 20 21:39:23 2022] PUT /identity/v3/projects/94662f66ac214075b6277e602fdb8c88/groups/305c4d70bd174764a1585a55545c6b3f/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:23.595087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540b04-277c-434e-8307-682949a12578 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.595448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540b04-277c-434e-8307-682949a12578 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.595652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540b04-277c-434e-8307-682949a12578 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.597085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 202/402] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:23.609569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24eefe-6729-4ef9-96a5-0e55259cd688 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.610152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24eefe-6729-4ef9-96a5-0e55259cd688 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.610403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24eefe-6729-4ef9-96a5-0e55259cd688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.629280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4d24eefe-6729-4ef9-96a5-0e55259cd688 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:23.630515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4d24eefe-6729-4ef9-96a5-0e55259cd688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:23.643228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:23.710508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 201/403] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:23.735255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540b04-277c-434e-8307-682949a12578 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:23.830192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d279cd-f162-4d30-b303-9cde307b23fd None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.830508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d279cd-f162-4d30-b303-9cde307b23fd None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.830736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d279cd-f162-4d30-b303-9cde307b23fd None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.832266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 202/404] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:23.837866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.838152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.838423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.848177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540b04-277c-434e-8307-682949a12578 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:23.869528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:23.871470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:23.897744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:23.898543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:23.898895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:23.899198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:23.908523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:23.914449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:23.924625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:23.925985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:23.926805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 203/405] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:23.938691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:23.989747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 203/406] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:24.019560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.123784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18db502-bfa7-4e35-a0ed-9b05cd1a4348 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.156957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:24.193779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.194447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.194726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.194997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.199895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:24.200399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:24.201987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:24.209052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5175cf-9fae-4820-8108-b7fb93a00794 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.211953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 204/407] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:24.218426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.270549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.271302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.271582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.271852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.284286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:24.290575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:24.307194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:24.308125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:24.308963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:24.311619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 204/408] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:24.314785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.334863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a3f1-750d-4760-bb49-f825440eee8c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.372726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734249bf-535f-498e-9b16-51945298831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.373934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734249bf-535f-498e-9b16-51945298831a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.374392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734249bf-535f-498e-9b16-51945298831a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.374842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734249bf-535f-498e-9b16-51945298831a None admin] PATH_INFO: `/v3/groups/b5d356c39aff40db8ecbb7a5be593e94` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.376243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734249bf-535f-498e-9b16-51945298831a None admin] RBAC: Authorizing `identity:get_group(group_id=b5d356c39aff40db8ecbb7a5be593e94)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:24.377919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734249bf-535f-498e-9b16-51945298831a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:24.380183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 205/409] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/groups/b5d356c39aff40db8ecbb7a5be593e94 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:24.386170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734249bf-535f-498e-9b16-51945298831a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.452535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e643bf-a8b5-4757-88a5-476b292c3775 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.487088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.487753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.488033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.488305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.489048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:24.489359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:24.490733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eba3f6-5a49-40df-bd3d-a1bec99cc7f2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:24.496046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734249bf-535f-498e-9b16-51945298831a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.500472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 205/410] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:24.501892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99d90-f89b-481a-8cfc-205a46955c25 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.502180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99d90-f89b-481a-8cfc-205a46955c25 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.502448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e99d90-f89b-481a-8cfc-205a46955c25 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.503713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 206/411] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:24.505161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:24.508071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e99d90-f89b-481a-8cfc-205a46955c25 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.526882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:24.536347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21d032-4743-4a73-be57-d909fbfc8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.537414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21d032-4743-4a73-be57-d909fbfc8610 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.537849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21d032-4743-4a73-be57-d909fbfc8610 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.538265 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21d032-4743-4a73-be57-d909fbfc8610 None admin] PATH_INFO: `/v3/projects/feb77f81bd3a4077b594b0faf70919df` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.539694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21d032-4743-4a73-be57-d909fbfc8610 None admin] RBAC: Authorizing `identity:get_project(project_id=feb77f81bd3a4077b594b0faf70919df)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:24.541286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21d032-4743-4a73-be57-d909fbfc8610 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:24.547959 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 206/412] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/projects/feb77f81bd3a4077b594b0faf70919df => generated 344 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:24.548434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.548730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.548993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.568835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:24.570222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:24.635974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e99d90-f89b-481a-8cfc-205a46955c25 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.655130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 207/413] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:24.656263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.691637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.693257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.693743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.694459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.696419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:24.698343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:24.701221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 207/414] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:24.702319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.785131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:24.798609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b338b-2f27-4126-a286-c03e9602ebda None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.829353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34733f6-698d-4f6b-bef5-2a65a7a5fd96 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:24.832370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:24.844003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.844924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.845254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.845531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] PATH_INFO: `/v3/projects/feb77f81bd3a4077b594b0faf70919df/groups/b5d356c39aff40db8ecbb7a5be593e94/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.847651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] RBAC: Authorizing `identity:create_grant(project_id=feb77f81bd3a4077b594b0faf70919df, group_id=b5d356c39aff40db8ecbb7a5be593e94, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:24.849084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:24.872840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 208/415] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Jul 20 21:39:24 2022] PUT /identity/v3/projects/feb77f81bd3a4077b594b0faf70919df/groups/b5d356c39aff40db8ecbb7a5be593e94/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:24.872840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:24.895695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:24.896526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:24.896806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:24.897079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:24.924467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:24.946550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:24.947575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:24.948378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:24.949495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 208/416] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:24.969265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.080207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd2adc-bfc0-480c-83e6-b5ba9664c267 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.132096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.133048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.133340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.133617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.133963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96390ec6-4cf4-4067-89a4-0fd46b02471b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.134863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:25.136612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:25.139615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 209/417] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:25.146417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.179369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:25.197273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.197958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.198241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.198516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.199308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] RBAC: Adding query filter params (name=member) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:25.199618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:25.201020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:25.215237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 209/418] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:25.224420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.250654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:25.338502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b98d4-5877-4f46-840c-767900943039 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.356329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdbfba-20a2-4f37-97d7-1cb2cacab085 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.392745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.393643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.394078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.394504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.414278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b88e5-f4fb-46a1-95ee-de57f7e248fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.415588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b88e5-f4fb-46a1-95ee-de57f7e248fa None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.416316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b88e5-f4fb-46a1-95ee-de57f7e248fa None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.416521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b88e5-f4fb-46a1-95ee-de57f7e248fa None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88/groups/305c4d70bd174764a1585a55545c6b3f/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.419086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b88e5-f4fb-46a1-95ee-de57f7e248fa None admin] RBAC: Authorizing `identity:create_grant(project_id=94662f66ac214075b6277e602fdb8c88, group_id=305c4d70bd174764a1585a55545c6b3f, role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:25.420730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b88e5-f4fb-46a1-95ee-de57f7e248fa None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:25.437504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 210/419] 10.222.0.86 () {58 vars in 1442 bytes} [Wed Jul 20 21:39:25 2022] PUT /identity/v3/projects/94662f66ac214075b6277e602fdb8c88/groups/305c4d70bd174764a1585a55545c6b3f/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:39:25.443091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f163b5-cff9-4244-8a49-119c041805f9 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.443091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f163b5-cff9-4244-8a49-119c041805f9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.443091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f163b5-cff9-4244-8a49-119c041805f9 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.447171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 211/420] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:25.448705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.448998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.449273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.469241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:25.471005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:25.493849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:25.532009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:25.533542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:25.566941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 212/421] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:25.596620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.654577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:25.724054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95c745-4498-48b3-a55c-a2aca66d494c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.749475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 210/422] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:39:25.754655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.772665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.773958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.774414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.774814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.787446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:25.796752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:25.797884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:25.798881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:25.806806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 213/423] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:25.807998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.881559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:25.900061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb104f3-2ef8-4c5b-879f-6f80a207a06e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.936044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc7690-a828-4437-8b23-d46b0d5bd42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.936936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc7690-a828-4437-8b23-d46b0d5bd42d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.937179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc7690-a828-4437-8b23-d46b0d5bd42d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.937452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc7690-a828-4437-8b23-d46b0d5bd42d None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.938640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc7690-a828-4437-8b23-d46b0d5bd42d None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:25.940121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc7690-a828-4437-8b23-d46b0d5bd42d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:25.941695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e015d6-ca12-4535-b3aa-6f52081ff89c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:25.941807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 211/424] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:25 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:25.943051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea288a18-acfc-426e-b9f9-3ed1b1a730d3 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.943354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea288a18-acfc-426e-b9f9-3ed1b1a730d3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.943630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea288a18-acfc-426e-b9f9-3ed1b1a730d3 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.944987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 212/425] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:25.946425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:25.949044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea288a18-acfc-426e-b9f9-3ed1b1a730d3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:25.990446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:25.991540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:25.992082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:25.993739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:25.995177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:25.995692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:25.997630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bf3a8-aa80-42bb-a98a-b8c3ea45a7f6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.010668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 214/426] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:26.012222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.012713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.013183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.035148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:26.037080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:26.063863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea288a18-acfc-426e-b9f9-3ed1b1a730d3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.107668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.108666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.109035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.109337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.110932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:26.112578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.114751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 213/427] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:25 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:26.119100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.151442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 215/428] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:26.152361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:26.154379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.236320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace87cd-17c7-4941-812f-93e28274d16a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.280632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.281696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.282184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.282673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.283702 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:26.293011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec386bfc-e29f-4c9b-ae08-200dacadb886 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.337018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.338177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.338655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.339160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.340341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:26.340968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:26.342762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.349633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 216/429] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:26.350651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.390112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:26.395173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.447335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:26.557957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee17b9f-0b36-44bb-a7f5-767be5099150 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.574251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 214/430] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:26.584194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.597254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.598357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.598834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.599443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.618011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:26.625849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:26.626760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:26.627638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 217/431] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:26.635017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.661925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:26.724123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cebf1-3cf2-4011-a9fd-021e38335e2b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.756595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:26.774123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.775210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.775502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.775779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] PATH_INFO: `/v3/projects/d7f0857875f64efaa76e5af8b36c50ba` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.776778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d3390-2607-4eb3-be6a-ef86b40aa775 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.777004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] RBAC: Authorizing `identity:get_project(project_id=d7f0857875f64efaa76e5af8b36c50ba)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:26.778475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.782222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 215/432] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/projects/d7f0857875f64efaa76e5af8b36c50ba => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:26.795115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.827958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.828886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.829606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.829890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.830656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] RBAC: Adding query filter params (name=member) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:26.830967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:26.832674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.845272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 218/433] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:26.853374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:26.933501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754efead-0893-4978-8ff4-45386b21d1bb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.967649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:26.978751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:26.979709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:26.980063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:26.980593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:26.982411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:26.984099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:26.986492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 216/434] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:26.991856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133caf03-379a-449f-a978-0866c8f8b4ed None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:26.993172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.043414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.045740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.045972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.047266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.082893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:27.130147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba486da0-3c32-48a3-8f87-5773957bfab1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.156651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:27.158114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:27.182606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.184140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.184944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.185429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.186957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=d7f0857875f64efaa76e5af8b36c50ba) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:27.187667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:27.189938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a749fd66-8fd6-4379-996e-65c2c491a6a6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:27.200438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 217/435] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=d7f0857875f64efaa76e5af8b36c50ba&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:27.262496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:27.294077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 219/436] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:27.299770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.300338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502217b-2582-4f4b-8700-f77840dc3826 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.301181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502217b-2582-4f4b-8700-f77840dc3826 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.301674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502217b-2582-4f4b-8700-f77840dc3826 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.303598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 218/437] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:27.309449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.310061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.310558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.330146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:27.331751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:27.418846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 219/438] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:27.436309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed142cd-02ea-4750-9b4c-767b7af049bb None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.443773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.480490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.481363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.481647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.481922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.486061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:27.488053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:27.489802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 220/439] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:27.495778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.522994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:27.562084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:27.581981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce368c-389e-40fa-95a6-7489e079cf05 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.626614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352591a-dc09-4df5-babc-5e02a1a2295a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.628176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.628176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.628453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.628725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.642663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:27.650714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:27.651649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:27.652645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 220/440] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:27.659165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.669884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.670978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.671435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.671849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.674198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:27.675932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:27.678894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 221/441] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:27.684919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.776084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce00d3-e4b3-48c4-95fb-8f24052d3964 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.791677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:27.802206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73150ea3-e62c-41bf-adb6-12392c63186d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.814776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.815482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.815764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.816038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.816792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:27.817100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:27.818488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:27.821365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:27.827789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 221/442] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:27.832992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:27.844925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.845748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.846035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.846296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.847243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=daae1e7e6e714a95b85fff4ea4b37745, scope.project.id=94662f66ac214075b6277e602fdb8c88) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:27.847556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:27.849013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd7778-ea98-4816-af74-2c226f5a47a8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:27.859117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 222/443] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=daae1e7e6e714a95b85fff4ea4b37745 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:27.859117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42fa9-7c14-42d4-8d9f-f8c48b5e50e3 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.859117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42fa9-7c14-42d4-8d9f-f8c48b5e50e3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.859117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42fa9-7c14-42d4-8d9f-f8c48b5e50e3 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.864247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 223/444] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:27.865659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.865951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.866235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.884882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:27.886158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:27.948793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d700f-4f93-4363-bf27-700b99b4c210 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:27.964962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 224/445] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:27.986778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:27.987587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:27.987831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:27.988113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:27.989195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.042915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.070021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.072085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.111272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f4598-2399-480c-b639-924dae0a9f9b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.126469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.155336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.156145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.156422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.156693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.169882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:28.178013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:28.178919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:28.180317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:28.181277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 225/446] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:28.186058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.189341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 222/447] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 21:39:28.194603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.278753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.293671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae9b8f-7d35-4704-909c-c12be5572994 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.301741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d265bcd-2a70-4829-a40a-1b1696038ad3 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.327228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.327892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.328176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.328451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.329202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:28.329513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.330889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.334437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.335307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.335586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.335859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] PATH_INFO: `/v3/projects/feb77f81bd3a4077b594b0faf70919df` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.337707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] RBAC: Authorizing `identity:get_project(project_id=feb77f81bd3a4077b594b0faf70919df)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.339169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.339644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 226/448] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:28.340836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 223/449] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/projects/feb77f81bd3a4077b594b0faf70919df => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:28.344924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.345974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.379753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.444431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593f86a-a796-49b4-9535-2448304f7715 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.444607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c28844-3277-4978-9a78-1a7fe3c85985 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.477506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.477707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.478164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.478441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.478532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.478712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.478802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.479093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] PATH_INFO: `/v3/groups/b5d356c39aff40db8ecbb7a5be593e94` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.480280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] RBAC: Authorizing `identity:get_group(group_id=b5d356c39aff40db8ecbb7a5be593e94)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.481735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.483400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 224/450] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/groups/b5d356c39aff40db8ecbb7a5be593e94 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:28.489331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.504447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.547739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.549117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.584360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.589471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4601e-10f3-4bb7-8790-881b8fcf0806 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.624814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.625469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.625750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.626026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.626862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] RBAC: Adding query filter params (group.id=b5d356c39aff40db8ecbb7a5be593e94, role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=feb77f81bd3a4077b594b0faf70919df) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:28.627203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.628632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.635027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 225/451] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/role_assignments?group.id=b5d356c39aff40db8ecbb7a5be593e94&scope.project.id=feb77f81bd3a4077b594b0faf70919df&role.id=62c943ce95d54a1999a447dca6a1103b => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:28.657799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 227/452] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:28.663647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.745961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:28.770289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d31829-996b-4fdc-886c-bf6196c319e3 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.806500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.807340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.807628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.807903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] PATH_INFO: `/v3/projects/94662f66ac214075b6277e602fdb8c88` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.809115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] RBAC: Authorizing `identity:get_project(project_id=94662f66ac214075b6277e602fdb8c88)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.810555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.812296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 226/453] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/projects/94662f66ac214075b6277e602fdb8c88 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:28.817838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:28.913928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8dcde-1633-47e8-8570-c3040f1aa4db None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:28.946296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92461f8-87b3-43e0-aacf-ce3831807a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:28.947159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92461f8-87b3-43e0-aacf-ce3831807a71 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:28.947435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92461f8-87b3-43e0-aacf-ce3831807a71 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:28.947703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92461f8-87b3-43e0-aacf-ce3831807a71 None admin] PATH_INFO: `/v3/groups/305c4d70bd174764a1585a55545c6b3f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:28.948852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92461f8-87b3-43e0-aacf-ce3831807a71 None admin] RBAC: Authorizing `identity:get_group(group_id=305c4d70bd174764a1585a55545c6b3f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:28.950285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92461f8-87b3-43e0-aacf-ce3831807a71 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:28.951909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 228/454] 10.222.0.86 () {56 vars in 1180 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/groups/305c4d70bd174764a1585a55545c6b3f => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:28.957853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:29.054924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b272-65a0-444c-bda4-616588b3231c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:29.088306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:29.088962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:29.089244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:29.089522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:29.090357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] RBAC: Adding query filter params (group.id=305c4d70bd174764a1585a55545c6b3f, role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=94662f66ac214075b6277e602fdb8c88) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:29.090672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:29.092152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2c88a-544b-4cc3-a7f3-be8e2901b664 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:29.097885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 227/455] 10.222.0.86 () {56 vars in 1376 bytes} [Wed Jul 20 21:39:28 2022] GET /identity/v3/role_assignments?group.id=305c4d70bd174764a1585a55545c6b3f&scope.project.id=94662f66ac214075b6277e602fdb8c88&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:31.884123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3543f-98d5-447a-af78-4a660586d0bd None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:31.884647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3543f-98d5-447a-af78-4a660586d0bd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:31.884647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d3543f-98d5-447a-af78-4a660586d0bd None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:31.886327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 229/456] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:31.891383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4f437-b7c8-4e35-856f-fe2b64c546e2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:31.891828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4f437-b7c8-4e35-856f-fe2b64c546e2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:31.892001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b4f437-b7c8-4e35-856f-fe2b64c546e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:31.908729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367789b-42cd-49ad-a4a8-525a93317bde None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:31.909031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367789b-42cd-49ad-a4a8-525a93317bde None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:31.909275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367789b-42cd-49ad-a4a8-525a93317bde None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:31.910642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 230/457] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:39:31.916086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:31.916369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:31.916645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:31.924375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:31.924509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-03b4f437-b7c8-4e35-856f-fe2b64c546e2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:31.925569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-03b4f437-b7c8-4e35-856f-fe2b64c546e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:31.930943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:31.946870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:31.948095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:32.023332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 228/458] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:32.025258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063839c-427d-4115-b2e1-d627d1ae1bf4 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.025819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063839c-427d-4115-b2e1-d627d1ae1bf4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.026379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063839c-427d-4115-b2e1-d627d1ae1bf4 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.028285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 229/459] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:32.035139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 231/460] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:32.047770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.049836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.050335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.050827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.075623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:32.076906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:32.150045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 230/461] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:32.150850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.175315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea9bc2-e217-4c30-ac0b-fc89e6213a63 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.219861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:32.223973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:32.226291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.228158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.228519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.228865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.241196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.242712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:32.261876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:32.262749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:32.264189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:32.264189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:32.264189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:32.264189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:32.264189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 232/462] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:32.264189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.264664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badea1d7-34c2-4eac-bc3f-bde484988ff6 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.303379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.304183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.304467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.304772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.316508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.317983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-5d3fbd03-5a21-487e-b741-c5b0a6101b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:32.337995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:32.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:32.339803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:32.339803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:32.339803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:32.339803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:32.339803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 231/463] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:32.340593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f28cc-1cfb-4608-b464-4b0cd83686d7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.340904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f28cc-1cfb-4608-b464-4b0cd83686d7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.341084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f28cc-1cfb-4608-b464-4b0cd83686d7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.346902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 232/464] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:32.346902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f28cc-1cfb-4608-b464-4b0cd83686d7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.381949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb29429-86e9-4825-aa54-7c2c24bf75f8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.428999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.429939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.430318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.430695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.446144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.448452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.464043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f28cc-1cfb-4608-b464-4b0cd83686d7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:32.472319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:32.473167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:32.474012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:32.474012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:32.474012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:32.474012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:32.474232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 233/465] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:32.475906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.490169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:32.506285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.507094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.507461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.508370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.509422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:32.509798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.511303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7debaf8b-2773-4f42-8819-dcdd800ee1ce None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.517075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:32.528924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 233/466] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:32.528924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.528924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.528924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.550515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:32.552169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:32.591948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427e046-189c-43ef-8e03-44a263147de6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.634962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.635662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.635946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.636224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.636956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:32.637267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.643145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 234/467] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:32.643145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.643348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.661905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 234/468] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:32.663165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.764114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc55d45-9949-44ea-9f3b-288c6faecbaa None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.776829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:32.794590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:32.802013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743112f4-a84c-42b3-b07e-fdb2680fbb76 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.803635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.804420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.805335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.805672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.806479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:32.806851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.808647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.821878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 235/469] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:32.822464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.849037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.850029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.850310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.850581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.852337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.854565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:32.885759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 235/470] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:32 2022] POST /identity/v3/users => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:32.886632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:32.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb3768-2647-4b38-9ed1-86892a39e096 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:32.981361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:32.982618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:32.983136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:32.983618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:32.984599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:32.986789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:33.003145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6229c91-9f38-4c2b-abb2-ebe0cad8e4e7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:33.013647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 236/471] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:32 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:33.015033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:33.052492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:33.070952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:33.072123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.072411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.072688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.090688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:33.092244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:33.114715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:33.116012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:33.118076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:33.118076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:33.118076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:33.118076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:33.118076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 236/472] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:33.124327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:33.131464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:33.175859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cfa8e-601e-4ca7-b92a-ad4ecc00f313 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:33.226314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:33.227981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.228495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.228972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.229887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:33.231575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:33.257460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 237/473] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:33 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:33.259932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986a4bb-9033-45f8-9d0f-23ca46c7aec9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:33.299973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:33.300575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.300857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.301129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.301880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:33.302192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:33.303628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47883f-30d3-4dfb-9f11-3583a9637238 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:33.319748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 237/474] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:33.325492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:33.342864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef9665-d11d-4600-b5e9-61f27216a778 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.343606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef9665-d11d-4600-b5e9-61f27216a778 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.344072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef9665-d11d-4600-b5e9-61f27216a778 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.347433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 238/475] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:33.354147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.354809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.355311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.374436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:33.376141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:33.436045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:33.447472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:33.459090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 239/476] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:33.465375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ca71-c650-4f65-a5fe-ffc14dd14cce None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:33.485405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:33.503544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:33.504371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.504658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.504939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.505659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:33.507461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:33.534750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 238/477] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:33 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:33.620446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e846426-093b-40be-b1dc-775f842fb574 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:33.675889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:33.676553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.676834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.677110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.677900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:33.679325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:33.689497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.sql.core [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 21:39:33.689497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 21:39:33.689497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [parameters: {'id': '9c28c007fc2b46f68f65d8e4b59a32b0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 21:39:33.689497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86636) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application conn.query(q) Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result.read() Jul 20 21:39:33.708166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 21:39:33.709103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application next(self.gen) Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application yield resource Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application next(self.gen) Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise self.value Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application session.commit() Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 21:39:33.710074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.session.flush() Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application compat.raise_( Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 21:39:33.711242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 21:39:33.712316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application conn.query(q) Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result.read() Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 21:39:33.713334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [parameters: {'id': '9c28c007fc2b46f68f65d8e4b59a32b0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:33.714327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 21:39:33.715438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:33.716491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 240/478] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Jul 20 21:39:33 2022] POST /identity/v3/roles => generated 144 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:33.723108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:33.836055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799b063-d280-4810-a9a8-747e25108b5c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:33.872987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e6838-922c-4b39-9a9d-419613690149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:33.873721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6838-922c-4b39-9a9d-419613690149 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:33.874010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6838-922c-4b39-9a9d-419613690149 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:33.874295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e6838-922c-4b39-9a9d-419613690149 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:33.881957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:33.888176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e6838-922c-4b39-9a9d-419613690149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-fe1e6838-922c-4b39-9a9d-419613690149 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:33.898503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:33.900112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:33.901421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 239/479] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:39:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:33.907967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:33.995185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:34.040777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18836bf-33bb-48bd-a057-9afca688280e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:34.090738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:34.091461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:34.091751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:34.092024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:34.092788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:34.093097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:34.094551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3b568-7fc8-4d58-b8db-d681f123b5f3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:34.111234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 241/480] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:35.454481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c17e3-0f56-489b-addd-b454bbeb67ff None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.454975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c17e3-0f56-489b-addd-b454bbeb67ff None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.454975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c17e3-0f56-489b-addd-b454bbeb67ff None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.456603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 240/481] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:35.462663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6720bf-6ec5-46e0-b540-63b297f1c637 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.463120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6720bf-6ec5-46e0-b540-63b297f1c637 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.463260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6720bf-6ec5-46e0-b540-63b297f1c637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.488043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ed6720bf-6ec5-46e0-b540-63b297f1c637 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:35.489248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ed6720bf-6ec5-46e0-b540-63b297f1c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:35.578852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 242/482] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:35.602509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c17e3-0f56-489b-addd-b454bbeb67ff None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:35.756356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c17e3-0f56-489b-addd-b454bbeb67ff None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:35.758870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc00a-78f8-42c9-ab56-f9adbb3a7f52 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.759188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc00a-78f8-42c9-ab56-f9adbb3a7f52 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.759469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdc00a-78f8-42c9-ab56-f9adbb3a7f52 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.761316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 243/483] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:35.766712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b836-93c4-4864-b18b-10c4f080c6f3 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.767134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b836-93c4-4864-b18b-10c4f080c6f3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.767983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b836-93c4-4864-b18b-10c4f080c6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.790444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-edf3b836-93c4-4864-b18b-10c4f080c6f3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:35.790444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-edf3b836-93c4-4864-b18b-10c4f080c6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:35.808070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:35.809098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.809524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.810312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.821815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:35.838355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:35.840463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:35.841788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:35.844961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 241/484] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:35.851253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:35.879915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:35.910110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 244/485] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:35.912525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321152-9d49-4568-a048-5aafc082780a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.912818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321152-9d49-4568-a048-5aafc082780a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.913100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321152-9d49-4568-a048-5aafc082780a None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.918978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 245/486] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:35.920286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:35.920578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:35.921226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:35.944517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:35.945953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:35.948013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:35.995248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb8c8d-e44b-472c-ab3b-ef8efb270e5b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.022073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 246/487] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:36.022942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.045027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.045854 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.046145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.046422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.047254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:36.047567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.049122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.064270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 242/488] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:36.072248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.137432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1277d-b7af-4f37-908a-99f3aa90e502 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.150779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.176954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.178043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.178471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.178890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.190572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b42391-2727-4af1-92a9-ca84ecd5e2d4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.191196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:36.200296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:36.201384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:36.202375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:36.204448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 247/489] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:36.205666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.212845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.229061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.229871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.230181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.230661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.243410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.244992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:36.263685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:36.264618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:36.265476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:36.265476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:36.265476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:36.265476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:36.269786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 243/490] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:36.270630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.325844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf1fd9-5e50-412f-bf3f-e3faf79b20e7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.364581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.365815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.366280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.366760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.379090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:36.380755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba66b0-c6ae-4f28-8259-c7173f6a3c5d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:36.388094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:36.389011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:36.389837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:36.392298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 248/491] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:36.393336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.423455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.424111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.424445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.424724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.425587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:36.425899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.427545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.429016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.438948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 244/492] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:36.440390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.481557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.511527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030c6a9-3fe3-4539-96c7-c4afc26952a7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.546915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.547775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.548125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.548397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.549118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:36.549422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.550804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.556416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9e5ac-3800-41fe-8d53-c98791263f1b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.563091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 249/493] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:36.563839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.598782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.599426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.599709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.599991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.600750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:36.601062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.603020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ee543-93b5-4874-8403-cafbf28f5e84 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.611553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 245/494] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:36.612821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec867e5-2e58-4791-9576-dd201c52d966 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.613109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec867e5-2e58-4791-9576-dd201c52d966 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.613389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec867e5-2e58-4791-9576-dd201c52d966 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.619232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 246/495] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:36.619232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec867e5-2e58-4791-9576-dd201c52d966 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.666147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.676502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fa3c16-2ee0-4d9a-8c70-faac5bc0e263 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.714278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.715085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.715372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.715645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.723740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.726373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.728828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.732332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec867e5-2e58-4791-9576-dd201c52d966 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:36.747457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:36.748479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:36.749389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:36.749389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:36.749389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:36.749389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:36.749389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 250/496] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:36.749686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.776430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.777235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.777522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.777804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.785459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.786912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-9a92477f-0e9f-45c9-9726-573b43f90764 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:36.801986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:36.803142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:36.804070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 247/497] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:36.804753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.804753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.804916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.821166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:36.822363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:36.852253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b4f3b-3732-4730-85f5-a6e81ebe5322 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:36.886679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:36.887506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.887793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.888072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.890237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 248/498] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:36.891079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.903722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:36.905193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:36.907130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-8cdca0aa-4bff-423e-8e68-381aa8b82d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:36.924705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:36.925775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:36.926929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:36.926929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:36.926929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:36.926929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:36.926929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 251/499] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:36.927232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efcb56-e63f-4756-a1f2-e3d1a4491e1c None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:36.927450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efcb56-e63f-4756-a1f2-e3d1a4491e1c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:36.927823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efcb56-e63f-4756-a1f2-e3d1a4491e1c None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:36.931294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 252/500] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:36.931294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efcb56-e63f-4756-a1f2-e3d1a4491e1c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:36.989262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.004847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092d109-73c4-4c5d-9a3a-62e77f3e3ab0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.032306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efcb56-e63f-4756-a1f2-e3d1a4491e1c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.038393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.040070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.040366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.040642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.041367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:37.041673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.043081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.054485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 249/501] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:37.056122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.066557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.067244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.067526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.067798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.068629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:37.068932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.070382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.108455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 253/502] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:39:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:37.109217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.158733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf5eb9-7daa-4ce1-9618-0fcfd7b4c2e6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.159878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.192918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a263a-ba82-4ff7-b15f-b9c382d7de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.193711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a263a-ba82-4ff7-b15f-b9c382d7de3e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.193990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a263a-ba82-4ff7-b15f-b9c382d7de3e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.194260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a263a-ba82-4ff7-b15f-b9c382d7de3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.204190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a263a-ba82-4ff7-b15f-b9c382d7de3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:37.207831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6eac3f-c49d-4593-b945-52ee9659024b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-119a263a-ba82-4ff7-b15f-b9c382d7de3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:37.212298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:37.213236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:37.214051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 250/503] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:37.214759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.215328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.215328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.225878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.232059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:37.233259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:37.245571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.246227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.246510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.246781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.247533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:37.247837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.249242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.260120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 254/504] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:37.260910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.300008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 251/505] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:37.300852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.359203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e039-2edb-4dbe-8d6a-824814540eca None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.372051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.393281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.394069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.394346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.394622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] PATH_INFO: `/v3/users/glance` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.402639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.404119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.406515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a9313-91cc-4418-8d0f-cb88cd12ede0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:37.420009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:37.420935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:37.421835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 255/506] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:37.422411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.442627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.443328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.443615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.443891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.447640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.513104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.514562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.523093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458cc15f-0efc-40bd-9fd2-a546fada384a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.556501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.557160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.557445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.557722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.558473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:37.558786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.560191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.568433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 256/507] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:37.568737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.602883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.618217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 252/508] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:37.619305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.654263 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.669462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22cf1a-8448-498c-ae58-235f5333c2d8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.702617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.703433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.703716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.704200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.711944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.713342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.719514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958cd37-eed1-4418-8bb2-39accfb9f8bc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:37.727454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:37.728525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:37.729467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 257/509] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:37.730034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.759255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.760079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.760338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.760669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.770697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.772196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:37.790070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:37.790950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:37.791871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:37.791871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:37.791871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:37.791871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:37.791871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 253/510] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:37.792146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.827200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.834081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3412c5-6f10-442d-b340-7e18c20ba148 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.875567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.876161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.876442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.876721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.877561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:37.877873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.879328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.883686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:37.895971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57befe4-15f6-4ac0-b1b5-16210b1ffdb4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:37.902153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 258/511] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:37.902153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:37.930335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:37.931154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:37.931442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:37.931720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:37.941933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:37.943439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:37.961504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:37.962526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:37.963522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:37.963522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:37.963522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:37.963522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:37.963522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 254/512] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:37.963806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.000811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf0065-6f5e-4ad1-992a-7e6b3d0d220c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.034395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.035215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.035499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.035772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.045997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.047493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.062086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531e79c-4425-4dde-ad06-4f867dfa5e22 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:38.065085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:38.066083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:38.067004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:38.067004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:38.067004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:38.067004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:38.067004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 259/513] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:38.067488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.072637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.097344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.098014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.098295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.098570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.099454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:38.099757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.101180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.126131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.138676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 255/514] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Jul 20 21:39:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:38.139637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.174413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcb6f8-28d2-422b-9826-402d509e5e88 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.207380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.208032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.208314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.208583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.209304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:38.209606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.210997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.223427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 260/515] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:38.224208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.241756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdaa6f4-0fa9-4e2d-a54b-506bd27440af None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.276189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.276828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.277107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.277385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.284175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.322819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9194da-3419-4a5c-ad0d-b53f9183adf1 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.342368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.346447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.347951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.356983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.357637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.357916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.358193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.358907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:38.359242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.360848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.372338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 261/516] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:38.373119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.446523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 256/517] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:38.447390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.472601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4091-6bfd-485b-b835-466b7976058a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.496350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.507209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.507903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.508340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.508636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.509363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:38.509675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.511039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.522483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 262/518] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:38.523308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.544979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.548988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a49631-58e5-4b49-a1c5-357c480e1dc0 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.582372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.583027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.583333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.583602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.623846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568384a-2f88-4d4a-9b59-43130e1d2bb7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.650641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.652134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.657987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.658651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.658930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.659237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.659945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.661379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.684049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 263/519] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:38 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:38.684855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.743441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.768418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:38.775105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 257/520] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:38.775699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.822884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c699f0d9-708c-4938-8f6f-e39275f63347 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.864075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.864877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.865158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.865430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.878549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.880986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.892126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1822c2-4b01-4d1b-8e02-e3bc464759fa None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:38.898466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:38.899626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 264/521] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:38.901462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:38.927399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:38.928187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:38.928464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:38.928729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:38.939269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:38.940744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:38.959412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:38.960325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:38.961423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:38.961423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:38.961423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:38.961423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:38.961423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 258/522] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:38.962220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.014873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.018065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a4ab8-f960-4e7d-a9df-89d6014bd280 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.018634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.055582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.056386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.056671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.056961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.063917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.065383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.076741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e139-b21a-4e7d-80cc-97a0bf951c64 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:39.081699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:39.082502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:39.083406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 265/523] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:39.084479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.121591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.122349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.122639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.122914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.135303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.136657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:39.154946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:39.156053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:39.156956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:39.156956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:39.156956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:39.156956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:39.156956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 259/524] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:39.157625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.198475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17a365-92c1-4a6d-9ad0-753651739a98 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.233949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.234785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.235355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.235628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.236474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:39.236783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.238190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.256174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 266/525] 10.222.0.86 () {56 vars in 1139 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:39.256991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.277259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.278372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de715ad-df7f-43c5-acde-51caff36c01b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.280771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.318989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.319674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.319954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.320781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.321510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:39.321813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.323208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.340522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 260/526] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:39.340522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.373880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614eee5-b146-416a-82ea-4ff4a71e0e80 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.410945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.411871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.412305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.412741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.413547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:39.413896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.415473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.435251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 267/527] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:39.436078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.448466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98735a-05a1-432f-a6f1-6701d2779e52 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.483580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.484265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.484541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.485082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.485816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:39.486116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.487510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.499921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 261/528] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:39.504993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.519511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.530317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.543144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03de16a5-dffb-4d2c-9051-2b67d6f0748a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.581664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.582308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.582590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.582862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.608199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dd3ac-46c5-48f0-a5a7-31b8733d211f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.645045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.645830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.646110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.646383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.653242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.653463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.654704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.654885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:39.669730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:39.670617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:39.671523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 262/529] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:39.671990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.741042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.764271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:39.767874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 268/530] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:39.768733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.787940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd537ed-f34e-4ccd-9ba7-1d1e36b4bd05 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.824860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.825535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.825826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.826098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.826959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=e06439f92e494fb4a68ffee7bdb0ffc5) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:39.827305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.828824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.843559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 263/531] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/role_assignments?user.id=e06439f92e494fb4a68ffee7bdb0ffc5&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:39.845184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.888834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1763381-2ecd-402e-8341-b385959e7346 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:39.929652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:39.930952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:39.931351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:39.931628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:39.939849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:39.941244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:39.960885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:39.962124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:39.962992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:39.967112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 269/532] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:39.967112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:39.996371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6674-e27f-4860-88e0-3bf7631b7ae2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.028116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:40.051656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.052848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.053551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.054380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.056529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:40.056991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.059594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.086766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 264/533] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:40.087873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:40.099690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31a179-b6d1-4f31-b928-367c70165281 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.104380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:40.137530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.138322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.138623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.138874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.151018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.152584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:40.172788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:40.173876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:40.174857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:40.174857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:40.174857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:40.174857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:40.174857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 270/534] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:40.175336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:40.235400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc405f6-9ef6-4caa-bd91-2ae5f30fa343 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.279700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:40.288387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.290047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.290480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.290902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.291902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:40.292075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ba28b-ccd4-4bd2-bcf6-1f0b49a3b9e3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.292506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.295304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.312184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 265/535] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:40.313262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:40.336970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.337983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.338440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.338923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.340452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=be5097840a23492585d297d35e2fea20) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:40.341171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.343293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.352220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 271/536] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:40 2022] GET /identity/v3/role_assignments?user.id=be5097840a23492585d297d35e2fea20&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:40.353494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:40.456236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:40.465901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f46837-2e70-48fe-b45b-a659c4717154 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.538129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.538797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.539501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.539926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.540656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:40.540969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.542335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.555300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 266/537] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:40.560677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:40.567251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5175814-eee1-46d8-8baf-0e73a614d9dc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.637670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.638708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.639253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.639732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.641054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=ee043775ebb54bed8642c9f735b46c7f) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:40.641573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.643316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.651661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 272/538] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:40 2022] GET /identity/v3/role_assignments?user.id=ee043775ebb54bed8642c9f735b46c7f&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:40.754090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83db1d-fe0f-453a-8f98-ef6af5fd3c63 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:40.795144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:40.805262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:40.806070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.806348 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.806623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.818165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:40.819611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-5d903b33-1e8e-4566-bd10-ee82928ae61d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:40.838802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:40.839764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:40.840636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 267/539] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:40.845774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:40.876713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:40.915107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e3fd8-3f23-4fe2-a1ec-8fce5d769f86 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.915445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e3fd8-3f23-4fe2-a1ec-8fce5d769f86 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.915737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e3fd8-3f23-4fe2-a1ec-8fce5d769f86 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.917992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 268/540] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:40.926909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:40.927248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:40.927526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:40.974204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:40.976229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:41.027891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb340bd-ac28-4420-9f17-dec7c496f8f4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:41.074641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:41.075723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:41.076237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:41.076812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:41.077946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:41.078463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:41.080282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:41.091097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 269/541] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:41.099108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 273/542] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:41.104900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:41.132149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:41.251101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df88841-10cb-4663-9283-edd5c84f69d0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:41.262350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:41.276867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:41.287583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8992e-7583-49b3-aa1a-730ed27648d9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:41.311772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:41.312454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:41.312732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:41.313009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:41.313909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=7f38c9f8b5ea438dbae61a15c1e84e1c) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:41.314159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:41.315711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:41.323212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 270/543] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:41 2022] GET /identity/v3/role_assignments?user.id=7f38c9f8b5ea438dbae61a15c1e84e1c&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:41.341848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:41.342902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:41.343355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:41.343764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:41.365627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:41.378823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:41.380011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:41.380980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:41.383798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 274/544] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:41.389649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:41.534189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22842244-2ac0-4f3e-8c5f-c2fcd45d22d3 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:41.572216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:41.572897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:41.573182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:41.573457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:41.574252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:41.574563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:41.576040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:41.585105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 271/545] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:41.591120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:41.746576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c7d97-8386-487e-ac6c-20f64367c2e0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:41.794160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:41.794963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:41.795274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:41.795549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:41.808697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:41.813720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:41.820998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:41.843510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:41.844431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:41.845269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:41.845269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:41.845269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:41.845269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:41.845905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 275/546] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:41.852603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:41.980909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:42.118491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89311075-448e-468b-b94e-36abfe6e5613 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:42.185212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:42.185904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:42.186193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:42.186471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:42.187255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:42.187569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:42.188971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5225a28-ec4e-41b2-8580-09a60006b1d6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:42.217136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 272/547] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:42.222662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:42.402467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2c719-29af-4497-b4e3-abf6ee9c0e70 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:42.434354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8aceb-094b-4a5f-8aa2-bcc9bd72bef0 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:42.434690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8aceb-094b-4a5f-8aa2-bcc9bd72bef0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:42.434921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8aceb-094b-4a5f-8aa2-bcc9bd72bef0 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:42.436739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 273/548] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:42.454289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:42.455122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:42.455412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:42.455679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:42.458131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:42.458417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:42.458698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:42.465428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:42.467175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:42.485539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:42.487257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:39:42.488678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:42.491154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 276/549] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:42.492862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:42.494158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:42.504415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:42.616571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88e84-0e6f-4ee4-bbf7-43f1f97078c6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:42.619651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:42.621046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 274/550] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:42.645416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:42.718309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:42.724243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:42.725269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:42.725767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:42.726774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:42.728112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:42.728632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:42.730304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:42.756373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 277/551] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Jul 20 21:39:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:42.762196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:42.850759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ffa8c-da6b-4474-bc10-2b6aa391ea39 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:42.900815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:42.902638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:42.903205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:42.904333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:42.919644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:42.928289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:42.929534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:42.931203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:42.939295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 275/552] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:42.941619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.021179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b78a9-92b6-4dc8-8c29-3fca1121171f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.049537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.077021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.082264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19a5b5-4a69-45db-8e8d-70fbe2ef39b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.083080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19a5b5-4a69-45db-8e8d-70fbe2ef39b7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.083381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19a5b5-4a69-45db-8e8d-70fbe2ef39b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.083760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19a5b5-4a69-45db-8e8d-70fbe2ef39b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.087090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e8a6a-5a1c-453b-813c-5b8d5791de36 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.123102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.123826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.124106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.124376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.125145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:43.125450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.126839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e436fa1-5a76-4a91-a389-a8aca542b85a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.138718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 276/553] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:43.138718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7213-8e4f-4154-9881-6b804ff64516 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.138718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7213-8e4f-4154-9881-6b804ff64516 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.138718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f7213-8e4f-4154-9881-6b804ff64516 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.142158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 277/554] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:43.143358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b679f-a835-4874-9ab1-c61c0544651e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.143645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b679f-a835-4874-9ab1-c61c0544651e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.143921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b679f-a835-4874-9ab1-c61c0544651e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.145285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 278/555] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:43.146434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b679f-a835-4874-9ab1-c61c0544651e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.161887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19a5b5-4a69-45db-8e8d-70fbe2ef39b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.163398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19a5b5-4a69-45db-8e8d-70fbe2ef39b7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.262810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b679f-a835-4874-9ab1-c61c0544651e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.283229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 278/556] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:43.283229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.283229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.283229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.299012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:43.300243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:43.302320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.306084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.307022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.307339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.307616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.318220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.319814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-3f8cef39-910c-4a61-a016-92812e82e8e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:43.337510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:43.338441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:43.339374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:43.339374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:43.339374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:43.339374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:43.339374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 279/557] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:43.340138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.340238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.340439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.352498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.358111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:43.359352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:43.381809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 279/558] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:43.382631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.443435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 280/559] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:43.444599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.501850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d1ad2-dd7f-4a96-9b10-e1d8d384f1b6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.538558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.539478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.539850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.540216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.546998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.547398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a31d28-a7b4-4bbd-a5a9-4768adbf1c43 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.552657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.554133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:43.572748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:43.573793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:43.574682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:43.574682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:43.574682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:43.574682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:43.574682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 280/560] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:43.574954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.590246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.590973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.591279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.591569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.592267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:43.592576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.593952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.596413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.614137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 281/561] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:43.615026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.682531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d849-d731-4d79-bdcd-ee7867012884 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.720355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.721145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.721423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.721698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.723807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afa5ff-763b-45e6-903c-30395cab263e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.732456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:43.740443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:43.741567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 281/562] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:43.747978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.768068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.769004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.769385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.769759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.780880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:43.788789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:43.790088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:43.791223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:43.796639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 282/563] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:43.796639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.809168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.850572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:43.866235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48481b-1a6b-40fb-bd54-d17c6b170889 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.903765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2142ad7-3187-4741-9dda-bf51e1a7650e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:43.905576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.906354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.906734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.907139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.908037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:43.908442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.909939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.922767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 282/564] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:43.923854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:43.942430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:43.943399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:43.943756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:43.944097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:43.953404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:43.954881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:43.969284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:43.970420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 283/565] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:43.976691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.034039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f368524-9561-47df-b4e7-75c3286767f7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.044996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.068659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d846546-7665-402f-add4-79bb78288fc0 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.070097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.071302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.071585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.071854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.072594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.072897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.074273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.084821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 283/566] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.084821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.092343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.111147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.111821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.112107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.112386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.113140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.113459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.114830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db53383-3ee0-404b-9928-736e75b8ea7f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.123531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 284/567] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.124747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebb4d2-7946-43bd-bad6-44cbd9cd6ab2 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.125034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebb4d2-7946-43bd-bad6-44cbd9cd6ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.125312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebb4d2-7946-43bd-bad6-44cbd9cd6ab2 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.128179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 285/568] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:44.128179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebb4d2-7946-43bd-bad6-44cbd9cd6ab2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.189155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b49537-92f3-4c10-a0c4-58c9e7f17b68 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.222200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.222989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.223299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.223574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.228134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebb4d2-7946-43bd-bad6-44cbd9cd6ab2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.230791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.232258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:44.247512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:44.248554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:44.249515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 284/569] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:44.250009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.267869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.268705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.269129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.269543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.270437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.270799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.272652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.283363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.292739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 286/570] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.293635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.328550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.354286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd37e3-f692-4dfe-97ef-62bd8ab8f77d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.389329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.390111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.390401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.390674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.392551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0c4d8-33dc-4601-914d-923c08d0a0cc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.400572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.402018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-88f3b76b-7c12-482b-9a88-e8357f55e9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:44.419753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:44.420825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:44.421911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:44.421911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:44.421911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:44.421911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:44.421911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 285/571] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:44.422264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.422446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.422792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.428238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.429037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.429325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.430152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.439807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:44.440150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.441006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:44.441605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:44.459181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:44.460100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:44.461242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:44.461242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:44.461242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:44.461242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:44.461242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 287/572] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:44.461530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.512117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 286/573] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:44.512954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.527046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.559543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.572521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3745b-80fd-407a-86e5-76f8ba00e79d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.607370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.607960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.608248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.608525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.609322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.609649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.611364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.614686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0a141-9173-499e-b375-0030ed86e024 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.622972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 288/574] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.623802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.654829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.655645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.655929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.656204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.666981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.668477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:44.685771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:44.687092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:44.688016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:44.688016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:44.688016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:44.688016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:44.688016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 287/575] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:44.688016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.725401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae403d-b38f-44f8-8c30-6222b2b17f34 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.752330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.758218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.758872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.759178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.759450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.760251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.760520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.762604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.774782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 289/576] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.775617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.787553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a409-0d31-4661-a25b-d18d6f53356a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.790227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.822066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.822725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.823010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.823314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.824042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.824350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.825711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.837603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 288/577] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.838396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.878013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37177196-45b4-4093-a0bf-b01bc1a7b1f9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.911191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.911983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.912266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.912543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.923817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:44.931599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:44.932712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:44.933563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 290/578] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:44.934267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:44.945314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b8a78-4d13-420b-af6a-c29e7c2d19c0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:44.970497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:44.981146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:44.981807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:44.982088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:44.982367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:44.983231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=6952f7293b2f46f28941b48429dd2e73) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:44.983544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:44.985233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:44.990591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 289/579] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/role_assignments?user.id=6952f7293b2f46f28941b48429dd2e73&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:44.991389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.008703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.049989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cff34e-4998-43dd-aa35-4ce55d151b96 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.098758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.099483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.099769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.100050 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.100797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:45.101114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.102741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.115578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc2aa13-f1d3-41d1-b227-b6d0b4238f8c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.116222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 291/580] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:45.117172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.160939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.161663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.161952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.162230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.170198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.171699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:45.186160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:45.187168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:45.188164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 290/581] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:45.188831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.240882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a29528-41d8-4244-9804-6c550a2c7202 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.247260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.275205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.285138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.285977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.286262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.286536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.295850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.297495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.302339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5c080-863b-4ee2-9cda-1435f41c9f62 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:45.313287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:45.314485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 21:39:45.315804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:45.319445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 292/582] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:45.319989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.346448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.347693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.348344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.348720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.349834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:45.350636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.353141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.368745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 291/583] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:45.368745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.429455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a4c4f-4615-4195-9308-fa31b3b194e5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.467901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.468931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.469494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.469795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.477806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.478832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dfd47-36e1-40ec-aa35-eaa23bed1582 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.479297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.492714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:45.493734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:45.494783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:45.495744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 293/584] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:45.496244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.520801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.521480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.521763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.522039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.524331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:45.524656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.526089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.549031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 292/585] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:45.549031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.558225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.619752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bd490-d415-4f48-8f47-d8c4596ab89f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.658219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.658431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bef6b9-739e-422b-89c6-46b495bb4e87 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.658944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.659269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.659547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.660392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:45.661287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.662195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.687531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 294/586] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:45.688083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.694745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.695859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.695959 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.696207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.707950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.709402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:45.728378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:45.729313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:45.730204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:45.730204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:45.730204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:45.730204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:45.730204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 293/587] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:45.730489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.756978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.802777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6221e5-7c8c-44a6-90f3-df5fef0ef109 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.810644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:45.839078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4ccc7-1c1d-4a15-a3b0-ec2e44f2c794 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:45.840553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.841328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.841604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.841895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.842680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:45.842983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.844436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.861705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 295/588] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:45.861705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.881339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:45.883092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:45.883539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:45.883822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:45.893820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:45.895289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:45.913536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:45.914778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:45.915806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:45.915806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:45.915806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:45.915806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:45.915806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 294/589] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:45.916081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:45.978378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebe25d-e4ed-4b98-91c6-bca19b835222 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.008662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.022022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.022821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.023899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.024187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.034899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.036382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.037409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658ef18-2cf7-42e7-883b-ede6adb0aeee None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:46.055528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:46.056511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:46.057419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:46.057419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:46.057419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:46.057419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:46.057419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 296/590] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:46.057703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.079797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.081387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.081744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.082119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.082226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.083507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:46.083993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.085591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.099125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 295/591] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:46.099869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.175754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b43e8-5afd-40ca-9551-637ba979f60a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.205083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b094b8-5e24-4476-bb4d-2abb77376f74 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.212583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.213475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.213758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.214032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/e06439f92e494fb4a68ffee7bdb0ffc5/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.229570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=e06439f92e494fb4a68ffee7bdb0ffc5, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.231207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.238797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.239572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.239929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.240214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.240939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:46.241255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.243248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.244891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.248601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 297/592] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:46 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/e06439f92e494fb4a68ffee7bdb0ffc5/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:46.249726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.263136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 296/593] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:46.263136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.362510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.380041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3819b9e-3591-4536-86de-635751e3abeb None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.402213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3ae7f-6867-47d2-891b-62a204c14fe2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.423845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.424900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.425337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.425755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.434929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.436445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.439991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.440714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.441000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.441277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.442013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:46.442317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.443735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:46.452611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:46.453573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 297/594] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:46.454960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.456814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 298/595] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:46.463888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.527378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.572840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8266ec-7937-4714-a44e-a1c738ebab13 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.578722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc8da1-ec50-44b4-9534-5b8513124ff3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.600295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.609234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.610363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.611017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.611017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.612181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:46.612654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.614196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.619887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.620894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.621317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.621732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.629448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.631227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.638503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 299/596] 10.222.0.86 () {56 vars in 1139 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:46.638503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:46.652855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:46.653963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:46.654896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:46.657376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 298/597] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:46.658318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.748414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bca702-5223-46e6-9a55-9c3da8f4c7f2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.770135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67237d0-a371-4174-b417-932b64d238d0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.786772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.790147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.791851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.792198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.792541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.802296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.804736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.811030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.812088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.812539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.812980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:46.821890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:46.822893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:46.823849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 300/598] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:46.824347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.828750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:46.830535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:46.849403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:46.850337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:46.851330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:46.851330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:46.851330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:46.851330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:46.857891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 299/599] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:46.858977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:46.882484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:46.960500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e483c8c-b8d5-4621-a8f1-8119704b3465 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.971542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a13805-f4ed-487e-8698-a507ec734332 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:46.998372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:46.999051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:46.999357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:46.999636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.000429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:47.000784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.002286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.010281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.011248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.011727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.012215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.013199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:47.013715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.015356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.021117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 301/600] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:47.022640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.033889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 300/601] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:47.035159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.066828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:47.133651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2310d5-fdce-4e0b-963b-68e1d2cae4d2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.141787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:47.151197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46539bc-1b15-4367-93f4-a90312fcdec6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.170325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.171321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.171816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.172821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.174028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:47.174542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.177074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1fa8d8-2027-47a0-a3e8-51034d222819 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.187123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.188002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.188279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.188550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/ee043775ebb54bed8642c9f735b46c7f/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.190971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 302/602] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:47.192245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bb3cb-1806-49c0-9e5f-88b904aeab1d None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.192536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bb3cb-1806-49c0-9e5f-88b904aeab1d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.192814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2bb3cb-1806-49c0-9e5f-88b904aeab1d None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.199538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 303/603] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:47.199538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bb3cb-1806-49c0-9e5f-88b904aeab1d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.210688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=ee043775ebb54bed8642c9f735b46c7f, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.212688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.228072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 301/604] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:47 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/ee043775ebb54bed8642c9f735b46c7f/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:47.229692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.319682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2bb3cb-1806-49c0-9e5f-88b904aeab1d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.326661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:47.361755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998683fe-b970-4fb8-8ceb-f9e66f8294a5 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.363106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e82b0-cbd4-41a3-8932-73bb1cae9b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.364450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e82b0-cbd4-41a3-8932-73bb1cae9b3c None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.364793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e82b0-cbd4-41a3-8932-73bb1cae9b3c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.365140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e82b0-cbd4-41a3-8932-73bb1cae9b3c None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/be5097840a23492585d297d35e2fea20/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.383184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e82b0-cbd4-41a3-8932-73bb1cae9b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=be5097840a23492585d297d35e2fea20, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.385105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e82b0-cbd4-41a3-8932-73bb1cae9b3c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.407159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 304/605] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:47 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/be5097840a23492585d297d35e2fea20/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:47.407159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.407159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.407159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.420112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.421259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.421739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.422161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.430998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:47.432534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:47.434908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.436720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.450253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:47.456368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:47.457864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:47.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 302/606] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:47.467566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.535635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 305/607] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:47.562435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.620204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ee6ad-177b-49ae-83d2-70cc3229e9fd None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.669016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.670623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.671141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.671901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.673151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:47.674097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.676311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.680912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:47.685162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cb4e6-ed65-4837-aad1-1672ea6ed8a8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.696459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 303/608] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:47.701540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.728649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.729856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.730341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.730818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.743185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:47.751650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:47.752593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:47.753543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:47.756310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 306/609] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:47.762402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:47.775295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:47.823291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5966cc-3dfc-4a9e-8445-b708466e1c22 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.868293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.869534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.870039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.870568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/7f38c9f8b5ea438dbae61a15c1e84e1c/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.887191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=7f38c9f8b5ea438dbae61a15c1e84e1c, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.889002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.900701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38ea74-fc30-46e6-84ea-f4b21c6c38cd None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:47.906638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 304/610] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:47 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/7f38c9f8b5ea438dbae61a15c1e84e1c/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:39:47.948884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:47.949582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:47.949905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:47.950351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:47.951294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:47.951698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:47.953347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:47.964734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 307/611] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:47.970331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:48.039609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:48.097140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac8174-92b8-4419-87db-d815c8ed4415 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:48.136061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:48.136767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:48.137046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:48.137310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:48.147875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:48.149485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:48.172656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:48.173647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:48.174594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:48.174594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:48.174594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:48.174594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:48.174594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 305/612] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:48.180699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:48.265962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:48.300900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c87b2c-9d23-4183-b121-2f3ac90b26e1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:48.338534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:48.339279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:48.339565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:48.339839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:48.340572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:48.340894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:48.342258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:48.359280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 308/613] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:48.365048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:48.520513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:48.526705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c48e2-7146-4810-a4ee-8c06b83c591f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:48.564022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:48.565187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:48.565652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:48.566069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:48.574797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:48.576430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-583f82ab-c608-4af3-9040-c2e018ca09e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:48.599390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:48.600553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:39:48.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:48.604335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 306/614] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:48.610524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:48.637903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fe6e5-638b-4168-a96e-945f564480bc None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:48.638409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fe6e5-638b-4168-a96e-945f564480bc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:48.638865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fe6e5-638b-4168-a96e-945f564480bc None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:48.640877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 307/615] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:48.647052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:48.647616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:48.648134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:48.680942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:48.685061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:48.814744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a92832-8d9a-4897-bcf5-e6b187cb811f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:48.845999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 308/616] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 21:39:48.869534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:48.883684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:48.884681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:48.885165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:48.885656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:48.886713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:48.887304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:48.888942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:48.905046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:48.909033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 309/617] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Jul 20 21:39:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:48.920358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.023107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:49.050259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f9872-b8bd-4210-8344-493bef214c8e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.052345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d2019-d56b-455d-a16b-e4863ca48ddf None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.135498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.136656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.137462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.138465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.138946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.139459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.140428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.140904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.160005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:49.169456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:49.170425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:49.171290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:49.171941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:49.173351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:49.174391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 309/618] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:49.180354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:49.214622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:49.215923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:49.216985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:49.216985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:49.216985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:49.216985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:49.219957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 310/619] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:49.225324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.308692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:49.346601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4479aa-2f7a-46fa-856f-c0be1df2a2ca None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.389486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.390906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.391422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.391898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.392898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:49.393424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:49.395004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:49.406326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 310/620] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:49.411626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.427119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:49.442106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb318e0-4adc-4099-9610-f2e8eddc7e6e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.515815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.516503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.516784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.517058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.517803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:49.518110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:49.519531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:49.532759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 311/621] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:49.534903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebabb15-51e6-404d-89d9-3264da24a295 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.540284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.575302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.576480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.576964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.577510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.591144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:49.593022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:49.612503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:49.613445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:49.614379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:49.614379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:49.614379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:49.614379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:49.616788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 311/622] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:49.621964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.693227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:49.739107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0260b30-55f8-4658-8aa9-3d8aafba8589 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.765916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:49.773230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9042305-ea15-47df-8df1-f98d7164ccf7 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.781118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.782136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.782622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.783123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.784057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:49.784624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:49.786230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:49.800248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 312/623] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:49.807653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.884023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.884665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.884948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.885223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.904193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:49.905670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b5f58fb2-867b-47dd-b4ed-788902b9d4bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:49.926220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:49.927238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:49.928146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:49.928610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 312/624] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:49.929930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9346b-f3ff-40ca-bcbe-5d84c723273a None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.930528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9346b-f3ff-40ca-bcbe-5d84c723273a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.930528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9346b-f3ff-40ca-bcbe-5d84c723273a None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:49.932017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 313/625] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:49.936592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9346b-f3ff-40ca-bcbe-5d84c723273a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:49.942660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230a0b4-f07e-443f-8975-ae3c9fc864fc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:49.974159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:49.991370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:49.993906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:49.994189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:49.994465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.004609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.006937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-df404226-ca6f-464d-bdfd-a188c8f9bd40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:50.027816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:50.028796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:50.029814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:50.031727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 313/626] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:50.031727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896598d7-8028-48c6-9e41-163249726927 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.031727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896598d7-8028-48c6-9e41-163249726927 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.032093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896598d7-8028-48c6-9e41-163249726927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.051563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9346b-f3ff-40ca-bcbe-5d84c723273a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.051787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-896598d7-8028-48c6-9e41-163249726927 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:50.052764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-896598d7-8028-48c6-9e41-163249726927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:50.100296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.101328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.101622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.101977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.102683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:50.102990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.104442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29b75-5d97-478d-ae5c-0e8508f3139e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.121520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 314/627] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:50.122819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc0a8-ad7d-43d8-81ba-e8457270f9cb None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.123127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc0a8-ad7d-43d8-81ba-e8457270f9cb None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.123584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394cc0a8-ad7d-43d8-81ba-e8457270f9cb None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.125010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 315/628] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:50.127775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 314/629] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:50.129700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cc0a8-ad7d-43d8-81ba-e8457270f9cb None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.132020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896598d7-8028-48c6-9e41-163249726927 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.150979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.237974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896598d7-8028-48c6-9e41-163249726927 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.241612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394cc0a8-ad7d-43d8-81ba-e8457270f9cb None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.252169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.276941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.277828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.278117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.278396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/6952f7293b2f46f28941b48429dd2e73/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.279589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.280425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.280856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.281129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.281964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:50.282270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.283715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1cad0-a6b8-4cf4-be80-893a6ebcf965 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.294243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=6952f7293b2f46f28941b48429dd2e73, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.295745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.303491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 316/630] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:50.305709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.305991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.306259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.313926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 315/631] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:39:50 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/6952f7293b2f46f28941b48429dd2e73/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:39:50.314754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.334356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:50.336401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:50.426272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 317/632] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:50.427710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.435094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d60a7-f900-48fa-a00c-1b1b89491ef8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.444537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.479396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.480291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.480812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.480949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.500805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:50.510083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:50.511299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:50.512385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:50.518992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 316/633] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:50.518992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.550241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.567940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3bdd5-a867-47e8-b505-8ae7a39e36f3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.612182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca989586-e1e4-4e16-a306-944ac82c2551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.613195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca989586-e1e4-4e16-a306-944ac82c2551 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.613473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca989586-e1e4-4e16-a306-944ac82c2551 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.614024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca989586-e1e4-4e16-a306-944ac82c2551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.637100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6a318-4fb9-404e-a691-a801a99b8e9b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.672137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.672947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.673234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.673513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.684901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:50.689791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca989586-e1e4-4e16-a306-944ac82c2551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.691281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca989586-e1e4-4e16-a306-944ac82c2551 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:50.692984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:50.694069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 317/634] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:50.695531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.705989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.784769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.805355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 318/635] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:50.805691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824f51a-c191-49f7-932b-08658e2b84ae None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.807300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef1074-e813-4e89-b4cc-61eca07d7b0e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.807789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef1074-e813-4e89-b4cc-61eca07d7b0e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.808256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef1074-e813-4e89-b4cc-61eca07d7b0e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.809960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 319/636] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:50.810846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef1074-e813-4e89-b4cc-61eca07d7b0e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.849768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.850842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.851470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.852085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.852948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:50.853257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.854625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.868780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 318/637] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:50.868780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:50.927614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef1074-e813-4e89-b4cc-61eca07d7b0e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.936416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:50.968327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:50.969252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.969533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.969806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:50.970710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:50.971351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:50.973228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd62cfd-6fe6-4970-8d1f-8ba9f9e10dce None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:50.977336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c40e6-86a8-411c-92cd-a265defddd95 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:50.988510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 320/638] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:50.988510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:50.988510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:50.988510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.008662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:51.010032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:51.012294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.013090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.013370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.013647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.024585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.026025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.028561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:51.045526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.046672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.047838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:51.047838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:51.047838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:51.047838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:51.048761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 319/639] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.049408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.099515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 321/640] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:51.099515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.161535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059185a-2edc-4645-b25d-fe25caa137e8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.196506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.198676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.198676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.198676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.209183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.210642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.218256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.224952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd96836-7ba0-41af-9003-6251bb9d894d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:51.229445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.230317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.236566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:51.236566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:51.236566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:51.236566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:51.236566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 320/641] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.236566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.264211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.265179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.265467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.265744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.275990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.277438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.285475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:51.296540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.297525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.298445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:51.298445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:51.298445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:51.298445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:51.298445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 322/642] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.298745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.346185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708abc8-e232-4a60-a317-fe67051404e5 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.382219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.382932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.383248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.383528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.384386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:51.384820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.386250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.399617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 321/643] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:51.400550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.407985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3034-0e9e-4952-b31c-e433f34d43fe None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.444886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.446078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.446583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.447184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.458612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:51.467860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.469097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:51.470166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:51.472494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 323/644] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.473532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.489815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.516183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f1fe0-aaec-4c3b-9b2a-c6e754044801 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.525948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.552606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.553274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.553613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.553894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.555242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:51.555558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.556923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.569045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 322/645] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:51.570052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.586118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bcd540-4dfa-4b02-b028-a0db8765550e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.622754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.623660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.624213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.624637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.625589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:51.626037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.627688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.647012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 324/646] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:51.647871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.682322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af97db-1397-41a5-b8ec-8eaefd3abd38 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.717144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.717859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.718143 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.718419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.719127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.725798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.728406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:51.743981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:51.745812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 323/647] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.746337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.757484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5b806-d6e5-4ad2-9d32-133097d11e3f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.775397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.796318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.796990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.797268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.797542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.798281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:51.798584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.799983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.812910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 325/648] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:51.812910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.856592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4bcf6-035b-42b7-8bc3-0b58567544e1 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.893672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.894471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.894742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.895019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.902283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.903756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.918491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55add37-ed89-48fd-97cd-22898840a006 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:51.919586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.920540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 324/649] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.922210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:51.954540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:51.961867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:51.962777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:51.963078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:51.963541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:51.973019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:51.975114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:51.994999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:51.996211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:39:51.997374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:51.998194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 326/650] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:51.999003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.031678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.047365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3963508-1bee-427e-997e-1f98e7a1e9e5 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.089267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.089930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.090246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.090485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.091305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:52.091614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.093032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.106332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 325/651] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:52.111812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.118461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58136abd-4a89-4348-9a12-361f09423e03 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.157830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.159184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.159497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.159872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.171645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.173102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:52.190142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:52.191151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:52.197415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:52.197415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:52.197415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:52.197415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:52.197415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 327/652] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:52.197415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.225760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47556c3-984c-426a-94a7-cfa2706c7cb7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.241246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.263455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.264155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.264441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.264712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.265556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:52.265872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.267637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.291311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 326/653] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:52.291311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.298451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.303839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9df4e5-1bd8-48cf-bbe4-33859a60842c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.342916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.343757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.344142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.344521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.345472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:52.345885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.347608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.370686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 328/654] 10.222.0.86 () {56 vars in 1137 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:52.370686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.404879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33316a29-0296-48bd-a3b7-425bfa96e78d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.442402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.443054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.443363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.443634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.444460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=e06439f92e494fb4a68ffee7bdb0ffc5) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:52.444766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.446220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.451960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 327/655] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/role_assignments?user.id=e06439f92e494fb4a68ffee7bdb0ffc5&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:52.452725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.482169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8316584-eaaa-4dc2-a887-546e85f7763c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.484138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.526208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.526885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.527191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.527481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.528212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:52.528514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.529926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.542650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.543023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 329/656] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:52.543916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.581147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93d82e-41e6-438f-bf19-583a76715b67 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.628578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb4d5e-e3f6-4473-bb95-a6bf20cb4dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.630797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4d5e-e3f6-4473-bb95-a6bf20cb4dc7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.630797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4d5e-e3f6-4473-bb95-a6bf20cb4dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.630797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb4d5e-e3f6-4473-bb95-a6bf20cb4dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.655485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13eb50-61da-48d6-b057-f3dd2264291a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.697593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a89edc-9585-4c31-a4bd-6de8ba534613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.698245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a89edc-9585-4c31-a4bd-6de8ba534613 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.698527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a89edc-9585-4c31-a4bd-6de8ba534613 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.698795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a89edc-9585-4c31-a4bd-6de8ba534613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.711531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb4d5e-e3f6-4473-bb95-a6bf20cb4dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.713067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb4d5e-e3f6-4473-bb95-a6bf20cb4dc7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.732255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.778509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a89edc-9585-4c31-a4bd-6de8ba534613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:52.780124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a89edc-9585-4c31-a4bd-6de8ba534613 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:52.790094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.828034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 328/657] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:52.829489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e581-ab58-4789-9f0e-91a645ffeb20 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.829776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e581-ab58-4789-9f0e-91a645ffeb20 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.830053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3e581-ab58-4789-9f0e-91a645ffeb20 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.831505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 329/658] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:52.832047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e581-ab58-4789-9f0e-91a645ffeb20 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.896455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 330/659] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:52.897758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.898078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.898357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:52.914177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:52.915527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:52.953842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3e581-ab58-4789-9f0e-91a645ffeb20 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:52.979223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:52.991253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 331/660] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:52.992759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:52.993755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:52.994734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:52.995201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:52.995621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.002945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.004444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:53.020157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:53.021172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:53.022153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 330/661] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:53.022773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.056988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.122005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f021ef-4e84-4969-b348-8f4fdd36096d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.139459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e7fc3-9e2a-47a3-ba73-351b615db5ae None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.160347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.161369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.161816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.162342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.174336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.174895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.175155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.175438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.175712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.177025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.186135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.187602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:53.196261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:53.197334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:53.198267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:53.198267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:53.198267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:53.198267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:53.198267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 332/662] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:53.198557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:53.213172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:53.214137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:53.215188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:53.215188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:53.215188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:53.215188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:53.215188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 331/663] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:53.215483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.253545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.305794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.315552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c1a51-f156-4b0f-bca2-7234c7fb1d6a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.328270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895b021-61fe-46d1-98e7-3aa37ed4fee0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.352042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.352841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.353123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.353402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.363911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:53.364622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.365488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.365950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.366399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.367507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:53.368001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.369601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:53.371982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:53.372940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:53.373818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 333/664] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:53.374539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.394444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 332/665] 10.222.0.86 () {56 vars in 1139 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:53.395619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.495888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c122c2-3230-4c19-becf-5fc772032daf None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.498987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.502843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b21f1-583b-4270-92a0-0da782336f2f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.535618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.536211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.536495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.536772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.537507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:53.537816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.539455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.539575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.540113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.540388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.540656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.541371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:53.541678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.543029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.551934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 334/666] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:53.552226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.553000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.563653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 333/667] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:53.563653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.661025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381df414-9b16-4f09-8eff-283c6b6f248d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.675320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18931e6c-a572-4fe1-b1c9-b58f691878db None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.697730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.698396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.698677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.698952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.699731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:53.700104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.701422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.714478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 335/668] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:53.714478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.715699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.716352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.716628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.716900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.743214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.786703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.789553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.791409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.820845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26288a-23c0-4878-b752-c3405a1e333e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:53.856176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:53.856956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:53.857240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:53.857517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:53.864464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:53.865911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:53.880575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:53.881632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:53.883550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:53.887103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 336/669] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:53.887959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.903414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 334/670] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:53.904461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:53.976892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:53.993185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf0aba-7c10-4c48-a638-05e2ae8e4e32 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.016488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.020227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfaa7a-6738-49ca-9550-bfdd3bf744e3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.028288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.029397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.029683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.029964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.037213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.039465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:54.054236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:54.055852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:54.056749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 337/671] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:54.057241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.064515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.065318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.065617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.065865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.078324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.079811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:54.100191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:54.101758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:54.102678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:54.102678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:54.102678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:54.102678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:54.102678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 335/672] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:54.102972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.177742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f9868-ae09-45af-ada2-cdb2d63d0d93 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.214679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.215793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.216088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.216358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.217157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:54.217472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.218130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dfdf4-4f59-4239-a200-1c085e09752e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.218917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.237641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 338/673] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:54.237641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.245414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.259704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.260505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.260787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.261071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.272350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.274005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:54.294613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:54.295579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:54.296504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:54.296504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:54.296504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:54.296504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:54.296504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 336/674] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:54.296782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.302728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.347553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45250f8d-a8b5-4745-8ba6-d3b97e123b1c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.386476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.387165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.387451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.387723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.388519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:54.388819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.390357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.402419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 339/675] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:54.403272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.413282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1c186d-00e6-4ab6-a4de-06075436d718 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.449507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.450169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.450444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.450715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.451468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:54.451772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.453142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.465073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 337/676] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:54.465875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.488262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.532238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9caac-ecce-4e4b-955e-ccaf373cc55a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.533264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.568315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.569349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.569867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.570238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.571090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=be5097840a23492585d297d35e2fea20) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:54.571419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.572859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.579215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 340/677] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/role_assignments?user.id=be5097840a23492585d297d35e2fea20&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:54.579999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf37f-2bba-4a14-917a-0205c936387e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.580122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.624252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.624932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.625219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.625498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.626238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:54.627104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.628564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.647232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 338/678] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:54.647232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.702158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9fc16-3ae5-4311-bb7d-7540241d2004 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.735422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.746586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.746586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.746586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.746586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.746983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=ee043775ebb54bed8642c9f735b46c7f) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:54.746983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.748606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c9079-9661-49e2-9b47-b8ff166beefd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.754284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 341/679] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/role_assignments?user.id=ee043775ebb54bed8642c9f735b46c7f&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:54.755609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0753b-e3a0-4d11-b83b-3ff5b0f15fd1 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.755853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0753b-e3a0-4d11-b83b-3ff5b0f15fd1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.756136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0753b-e3a0-4d11-b83b-3ff5b0f15fd1 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.757617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 342/680] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:54.761534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0753b-e3a0-4d11-b83b-3ff5b0f15fd1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.769445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46b36e-23e8-425b-a592-ff732db87384 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.800892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:54.820936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.821722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.821989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.822266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] PATH_INFO: `/v3/users/swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.831652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.833180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-8d6cb304-c060-4ed0-b141-4ce70dd84f2b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:54.851757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:54.852794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:39:54.853767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:54.856171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 339/681] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:54.857673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.857941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.858212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.877131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:54.878353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:54.888390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0753b-e3a0-4d11-b83b-3ff5b0f15fd1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:54.937616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:54.938788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:54.939331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:54.939810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:54.947505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:54.949238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:54.960475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 340/682] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:54.961395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:54.964957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:54.965930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:54.966837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:54.969337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 343/683] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:54.976566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.040665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:55.095780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c47bb0a-d5a2-4306-bbd0-b30e1d5706c7 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.109673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff3486-fbd3-4a92-9a7d-b5e63e6e79d3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.130731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:55.136914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.137587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.137869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.138141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.138937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:55.139278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.140713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:55.153898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 344/684] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:55.154663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.167185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.168335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.168841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.169373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.170485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:55.171001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.172884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:55.199222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 341/685] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Jul 20 21:39:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:55.199222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.274352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a5f90-b22c-4e32-9131-deec695cea1f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.288570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:55.309602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.310401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.310681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.310952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.316636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20cf3f-cb5e-4faa-b049-286fe882dea8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.322365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:39:55.331758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:55.332708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 345/686] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:55.334332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.369372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.370414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.370902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.371558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.373417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=7f38c9f8b5ea438dbae61a15c1e84e1c) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:55.373960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.375627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:55.385041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 342/687] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/role_assignments?user.id=7f38c9f8b5ea438dbae61a15c1e84e1c&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:55.386304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.430189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:55.513934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1af4f-25aa-49c2-97b0-3028a25fcf24 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.541038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180eb78-f9bd-4a41-9f46-0db90a9e5363 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.557126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.558193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.558707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.559261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.560236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:55.560766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.563266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:55.574357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 343/688] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:55.580082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.622655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.623325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.623607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.623878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.683445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:55.711630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:55.714715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f65771-a823-4879-be1a-0d7a8f4effc0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.715822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.717796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:55.758073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.758870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.759186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.759466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.772887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.775146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:55.795013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:55.795982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:55.796932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:55.796932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:55.796932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:55.796932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:55.797421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 344/689] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:55.802388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.878340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 346/690] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:55.884497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:55.947232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42431232-d2a1-446d-a6eb-e01053bac611 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:55.985475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:55.986044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:55.986328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:55.986605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:55.987493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:55.988096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:55.989294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:56.002924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 345/691] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:39:56.008525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:56.021302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:56.045102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:56.084442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ac322-13d5-45ca-a613-6eceb1b3ded6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:56.122117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:56.123315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:56.123663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:56.124011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:56.135327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:56.136822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:56.158413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:56.159404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:56.160403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:56.160403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:56.160403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:56.160403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:56.165666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 347/692] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:56.170768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e50c42-19bc-48db-aa73-201ed47c5c3d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:56.172709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:56.224105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:56.225430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:56.228432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:56.228941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:56.241939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:56.243856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:39:56.270572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:56.271648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:39:56.272597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:56.278057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 346/693] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:39:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:56.283075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:56.367752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ebeb3-5e73-474f-8dd9-d9c3e1d349bc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:56.399749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:56.432469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:56.437043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482af5d-63dd-408a-aa4c-7ab63f9b07c7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:56.464294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:56.464980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:56.465349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:56.465532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:56.466262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:56.466570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:56.467997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:56.481436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:56.482433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:56.482910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:56.483415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:56.484446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:56.484949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:56.486963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:56.495020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 348/694] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:56.500750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:56.524791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 347/695] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:39:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:56.530293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:56.676662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d53db7-44d0-4635-8ccc-d65e438e585e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:56.713570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3daf-6ea2-453e-8eea-44f7f0d3c47c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:56.734132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:56.734931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:56.735509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:56.735784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:56.759701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:56.765900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:56.809000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:56.815036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:56.816003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:56.816942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 349/696] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:56.822295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f51334-7020-42f0-bcf4-588abb795f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:56.823452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51334-7020-42f0-bcf4-588abb795f8e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:56.823948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51334-7020-42f0-bcf4-588abb795f8e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:56.824445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51334-7020-42f0-bcf4-588abb795f8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:56.826473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:56.957172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:56.973551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f51334-7020-42f0-bcf4-588abb795f8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:56.977404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f51334-7020-42f0-bcf4-588abb795f8e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.028244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91f0db-cf71-42ab-91bd-6b01454005b6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.073622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.074565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.074991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.075465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.076566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:57.077177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.078366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefd4e0-799c-4fa0-b6bb-66a1c3f97f32 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.100601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 350/697] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:57.102670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac0431-aa7c-4757-b3e0-72f38fcabcd5 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.102955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac0431-aa7c-4757-b3e0-72f38fcabcd5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.103254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac0431-aa7c-4757-b3e0-72f38fcabcd5 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.108629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 351/698] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:57.110420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac0431-aa7c-4757-b3e0-72f38fcabcd5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.153331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 348/699] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:39:57.155664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.156182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.157085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.181088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:57.183964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:57.210133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:57.219014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac0431-aa7c-4757-b3e0-72f38fcabcd5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.227393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:57.258785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.259478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.259759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.260032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.260748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 349/700] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:57.260857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=6952f7293b2f46f28941b48429dd2e73) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:57.261158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.262099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.262594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599f12-4dec-48d7-9969-7f4b1a775b70 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.272775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 352/701] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/role_assignments?user.id=6952f7293b2f46f28941b48429dd2e73&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:57.274202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc8057-4f71-4e92-b64d-5a873edcf341 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.274486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc8057-4f71-4e92-b64d-5a873edcf341 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.274762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc8057-4f71-4e92-b64d-5a873edcf341 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.280693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 353/702] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:57.282724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.283013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.283372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.305130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:57.306353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:57.403672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 354/703] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:57.403672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.427030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb75cf0-8a62-4a11-a0ed-b187a8215cc4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.479183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.480223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.480651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.481034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.493298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.494926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:57.515554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:57.516516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:57.518212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:57.518212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:57.518212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:57.518212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:57.518212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 350/704] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:57.518212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.532476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:57.539752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:57.545522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889fbc5-bde5-420d-a68e-5c5459835b3e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.590554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.591709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.592257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.592539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.606778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.608701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:39:57.630375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:57.631445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:57.632982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:39:57.632982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:39:57.632982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:39:57.632982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:39:57.632982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 355/705] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:57.637534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250e446-037c-4294-9edc-f25686dd3ad2 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.638585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.683631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.684521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.685220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.685511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.686472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.688162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:57.708280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:57.709298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:39:57.710148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:57.711586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 351/706] 10.222.0.86 () {56 vars in 1129 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:57.712012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.761815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af26a3-5c1a-479a-b89d-b3cd62e3ff64 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.804580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.805257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.805541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.805817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.806556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:57.806860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.808284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.810012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:57.822217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 356/707] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:57.823342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.842967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:57.849111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889d6b8-b1bf-44cd-b470-a86fc4cb01ba None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:57.901783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:57.902453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:57.902736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:57.903009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:57.903764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:57.904075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:57.905455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:57.920256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 352/708] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:57.921097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:57.968587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a1b34-3524-4c1e-8e30-48d18229e60c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.008568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.009240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.009525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.009804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.010518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:58.010826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.012301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.017469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 357/709] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:58.018162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.047318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef4a5f-7014-483d-a1d6-a570dd65b531 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.105993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.106795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.107376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.107376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.108006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:58.111111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:58.117168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.119503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.133198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade7f50-0c2d-4f64-b50f-20244d77937f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:39:58.135005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:58.136258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 353/710] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:39:58.137206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.175937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd60b5-693a-4baa-b313-d133af3a8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.176616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd60b5-693a-4baa-b313-d133af3a8664 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.176900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd60b5-693a-4baa-b313-d133af3a8664 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.177176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd60b5-693a-4baa-b313-d133af3a8664 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.177884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd60b5-693a-4baa-b313-d133af3a8664 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.179442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd60b5-693a-4baa-b313-d133af3a8664 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.199218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 358/711] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:39:58 2022] POST /identity/v3/users => generated 254 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:39:58.200936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719d1f2-16ec-4f93-b830-f3632f9da91c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.201442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719d1f2-16ec-4f93-b830-f3632f9da91c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.201930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719d1f2-16ec-4f93-b830-f3632f9da91c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.203472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 359/712] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:58.204447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719d1f2-16ec-4f93-b830-f3632f9da91c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.259046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1fbf0-a7cc-437d-bf55-613471d8a2c0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.311727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.312322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.312608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.312889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.313622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:58.313927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.316050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f04e4f-65ae-44e6-9e9e-df41c7436b66 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.325407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 354/713] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Jul 20 21:39:58 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:58.327509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.327812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.328094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.332206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719d1f2-16ec-4f93-b830-f3632f9da91c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.352136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:58.353686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:58.370829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:58.387381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.388448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.388971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.389521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.390686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:58.391245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.399379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.423697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 360/714] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:39:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:58.431220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.436852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:58.453979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 355/715] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:58.489919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.570963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafda192-b010-4367-8089-b1df57ac19fb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.623170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d686ce0-2754-4720-b41f-ec0dd038cfdd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.626679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.627670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.628162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.628654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.629698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=e06439f92e494fb4a68ffee7bdb0ffc5) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:58.630262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.633314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.641877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 361/716] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:39:58 2022] GET /identity/v3/role_assignments?user.id=e06439f92e494fb4a68ffee7bdb0ffc5&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:58.667460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.668272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.668558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.668833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] PATH_INFO: `/v3/services/network` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.669511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.670905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:39:58.684093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:39:58.685194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:39:58.686047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 356/717] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:39:58 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:39:58.693147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.783200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:58.847173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb035b43-23c3-4b36-ac36-dba5c3dc8f71 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:58.900919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:58.901615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:58.901899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:58.902174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:58.902902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] RBAC: Adding query filter params (name=network) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:58.903236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:58.904627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7a93f-6501-468a-8953-41a8e73e9df5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:58.911318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 362/718] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:39:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:58.917495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:58.984062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:39:59.037449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12caf4-3f3b-443a-8938-f044d3cbb40f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:59.083178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:39:59.089541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:59.090117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:59.090621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:59.091620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] RBAC: Adding query filter params (type=network) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:39:59.099256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:39:59.100982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb78646-2aba-4249-94d6-d15145cb2136 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:39:59.106954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 357/719] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:39:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:39:59.663488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a5d5-bbcc-4c23-a5dd-bc615200d2d8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:59.663920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a5d5-bbcc-4c23-a5dd-bc615200d2d8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:59.664060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270a5d5-bbcc-4c23-a5dd-bc615200d2d8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:59.665658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 363/720] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:39:59.671306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:39:59.671800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:39:59.671881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:39:59.692105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:39:59.693320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:39:59.795145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 358/721] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:39:59.830849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a5d5-bbcc-4c23-a5dd-bc615200d2d8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:39:59.970180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270a5d5-bbcc-4c23-a5dd-bc615200d2d8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:39:59.996622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:00.019872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:00.020674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.020955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.021229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.041990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:00.049656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:00.050631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:00.051547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:00.052219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 364/722] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:00.056853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:00.135088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:00.216967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66069b6-86fb-4794-8466-b4c1ec560d22 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:00.263874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:00.264549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.264833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.265105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.265858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:00.266170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:00.267574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:00.279479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 359/723] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:00.284607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:00.414110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591f933-b5f1-45f6-ab99-d67a17a875be None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:00.459860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:00.460920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.461241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.461521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.478735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:00.480235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:00.500076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:00.501026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:00.502045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:00.502045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:00.502045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:00.502045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:00.502045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 365/724] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:00.512377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:00.653394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd99ba-e228-4ff5-b967-f1061201f157 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:00.670551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:00.705059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:00.705822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.706007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.706269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.707001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:00.707337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:00.708716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f829b-dbf9-4550-8ecf-a475e7fe2158 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:00.722204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 360/725] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:00.727879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:00.772891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:00.828279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eba4b-c86d-42c5-99af-6755f0f9d63b None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.828589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eba4b-c86d-42c5-99af-6755f0f9d63b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.828836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290eba4b-c86d-42c5-99af-6755f0f9d63b None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.830277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 361/726] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:00.838736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87397d-003e-4a28-85a2-7ecf385b3d38 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.839022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87397d-003e-4a28-85a2-7ecf385b3d38 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.839326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87397d-003e-4a28-85a2-7ecf385b3d38 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.840630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 362/727] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:00.842370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2032f-f7d7-4ca7-82fc-11bd2ee1baac None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.842657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2032f-f7d7-4ca7-82fc-11bd2ee1baac None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.842938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2032f-f7d7-4ca7-82fc-11bd2ee1baac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.862178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-dfc2032f-f7d7-4ca7-82fc-11bd2ee1baac None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:00.863522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dfc2032f-f7d7-4ca7-82fc-11bd2ee1baac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:00.873001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b477b26-b2f2-445c-a6c2-60650badaaee None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:00.909685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:00.910863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.911346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.911913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.921109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:00.922973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:00.939502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:00.940491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:40:00.941489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:00.943359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 363/728] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:40:00.944012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 366/729] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:00.944722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:00.945027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:00.945304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:00.952430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:00.966819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:00.974148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:01.055552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 364/730] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:40:01.056656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.064015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.073178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a8fc-b8de-4ce3-a4b7-cf41f13a9da9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.095546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.120557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.121227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.121501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.121770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.122607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:01.122910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:01.124384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d810a2-c259-42c3-a8c2-1fc774b73b88 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:01.146344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 367/731] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Jul 20 21:40:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:01.146344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40df0-06eb-421b-80a9-dc50296f7c1c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.146344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40df0-06eb-421b-80a9-dc50296f7c1c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.146344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc40df0-06eb-421b-80a9-dc50296f7c1c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.146344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 368/732] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:01.146344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc40df0-06eb-421b-80a9-dc50296f7c1c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.181479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b242e1-21ff-474e-a319-bd42a0ea3c81 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.218963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.219683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.219968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.220246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.220996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:01.222422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:01.240669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 365/733] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:01 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:01.252147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.272007 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc40df0-06eb-421b-80a9-dc50296f7c1c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.312966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad63c9-d7cd-4fb1-84ad-4b10777bd6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.313796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad63c9-d7cd-4fb1-84ad-4b10777bd6ef None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.314081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad63c9-d7cd-4fb1-84ad-4b10777bd6ef None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.314360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad63c9-d7cd-4fb1-84ad-4b10777bd6ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.326949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad63c9-d7cd-4fb1-84ad-4b10777bd6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-69ad63c9-d7cd-4fb1-84ad-4b10777bd6ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:01.334980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:01.336773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:01.338371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 369/734] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:01.339157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.339157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.339157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.352319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.360800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:01.361870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:01.401586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.417778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1293-c45f-4383-abe9-65f03b365bca None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.447903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 370/735] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:40:01.448514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.456434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.457094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.457890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.458175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.552670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:01.555284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:01.563937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720a61e-a7f0-4b32-85cf-fae70e43b318 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.600872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.601785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.602082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.602362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.603230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:01.603538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:01.604994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:01.607648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.616047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 371/736] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:01.617276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.679428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.699341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 366/737] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 21:40:01.700471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.729768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870ea09-afae-48ae-8208-4fb5340bde1a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.767722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.768451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.768732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.769009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.781202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:01.789112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:01.790172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:01.791053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:01.795385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 372/738] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:01.796237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.830913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffb8f-401b-45fc-af86-629cd83e1bf5 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.864378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:01.883023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.883860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.884145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.884422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.901116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:01.903487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:01.920235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac789d-a4ce-4cef-9c9f-515a378ed1ae None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:01.926957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:01.927885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:01.928788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:01.928788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:01.928788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:01.928788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:01.937873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 367/739] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:01.937873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:01.970528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:01.971309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:01.971596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:01.971903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:01.987884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:01.989399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:01.994496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:02.016207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:02.017165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:02.018084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:02.018084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:02.018084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:02.018084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:02.020131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 373/740] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:02.021133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.058819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68536d3d-1189-4686-8b6f-1e2a32239d09 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.095018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.095748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.096034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.096315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.097078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:02.097390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.098770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.107978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 368/741] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:02.108827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.138340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6ca02-ffbf-4ea9-b71f-1e145f7dc870 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.150491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.179678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.180753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.181258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.181726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.182755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:02.183299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.184941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.197663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 374/742] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:02.198747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.215173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4cc5a-c1e8-449e-82b2-2aeb60231983 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.216720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.253433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.254095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.254380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.254656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.255430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:02.255934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.257640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52fb88-de39-4517-9045-e1842f05ba21 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.275458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 369/743] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:02.276900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f214dd-59e6-4d11-80b0-0925d8bae5b5 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.277190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f214dd-59e6-4d11-80b0-0925d8bae5b5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.277471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f214dd-59e6-4d11-80b0-0925d8bae5b5 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.282629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 370/744] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:02.282629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f214dd-59e6-4d11-80b0-0925d8bae5b5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.306599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742c558-6ff4-4d4c-a088-748d27f06b21 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.342276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.343080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.343362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.343633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.354622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.356197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:02.374683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:02.375715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:02.376621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:02.376621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:02.376621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:02.376621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:02.379210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 375/745] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:02.380507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.390719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f214dd-59e6-4d11-80b0-0925d8bae5b5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.404850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.427991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.428806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.429129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.429674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.436905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.439053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-e0fe87b8-c524-4a4b-a5cc-8b6b9e1a6b68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:02.454766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:02.456012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:02.457120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 371/746] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:02.460900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.461388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.461932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.483707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:02.485919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:02.489842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a68223-709b-4308-a106-0ebf6de7258a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.504819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.526179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.526987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.527314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.527589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.534591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.536083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:02.558477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:02.559716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:02.561047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:02.571509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 372/747] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 21:40:02.571509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.571705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 376/748] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:02.571705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.661079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.672398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15385-73e1-4afd-93b6-c44d290066fb None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.681730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce016762-90d5-46f7-bca0-6295be7f9757 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.707766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.708421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.708707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.708986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.709708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:02.710012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.711464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.717986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.718875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.719457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.719739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.720585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:02.720902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.722324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.727560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 373/749] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:02.727560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.740250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.751362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 377/750] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:02.751362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.833573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcb8d9-037c-46e5-bf1d-14294c0016c6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.856990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec563c3f-0d76-4f00-a1d9-1bd4985c7c2b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:02.869807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.870476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.870763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.871040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.871871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:02.872181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.873619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.892140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 374/751] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:02.892140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.901689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:02.902364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:02.902648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:02.902926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:02.903706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:02.905078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:02.916155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:02.921900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 378/752] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:02 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:02.922808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:02.999297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.048815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8991b5b-cbf7-4ecd-ad64-cecff1187863 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.057818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02df8b-79ff-4ae5-b6b4-2c9ae832195c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.101759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.102472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.102814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.104063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.104170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.104562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.105900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.105900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.117489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.119437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:03.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:03.136975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:03.137988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:03.140254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 375/753] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:03.141296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.184201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.194001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.196207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.259329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd212e-ec71-42aa-ab22-e2fe0651fb9e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.293513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.300243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.301141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.301622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.302269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.303765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=6952f7293b2f46f28941b48429dd2e73) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:03.305053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.306889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.320773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 376/754] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/role_assignments?user.id=6952f7293b2f46f28941b48429dd2e73&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:03.320773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.334600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 379/755] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:03.339077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.497698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.506634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432f210-96fd-4ef7-89b3-5fdcd54abf08 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.515815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4416970-9f8e-43d8-bdba-1a2719423898 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.550339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.551597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.552085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.552584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.562912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.563654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.563943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.564353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.565359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:03.565673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.565872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.567132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612f3fb-4e06-4f5f-9870-a5e1a88d3be8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.567820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:03.591186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:03.592227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:03.593798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:03.593798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:03.593798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:03.593798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:03.599876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 377/757] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:03.599876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02409-2d9b-42a1-87eb-e18c5dc9e099 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.599876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02409-2d9b-42a1-87eb-e18c5dc9e099 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.599876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02409-2d9b-42a1-87eb-e18c5dc9e099 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.600470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 380/757] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:03.600782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 378/758] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:03.601871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.602052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb02409-2d9b-42a1-87eb-e18c5dc9e099 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.664526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.715524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb02409-2d9b-42a1-87eb-e18c5dc9e099 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.745457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80c24c-8e03-4d2b-b948-2c78eb25fe71 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.753840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.754783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.755321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.755799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.756724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:03.757779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.759680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55946fee-957c-4110-9e1d-f679de332719 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.773854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 379/759] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:03.777243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a4178-683d-42f2-a681-a00941dba48f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.777748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a4178-683d-42f2-a681-a00941dba48f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.778224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a4178-683d-42f2-a681-a00941dba48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.793301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:03.794113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:03.794398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:03.794675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:03.797354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-108a4178-683d-42f2-a681-a00941dba48f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:03.798773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-108a4178-683d-42f2-a681-a00941dba48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:03.806966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:03.808992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:03.822215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:03.834580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:03.835762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:03.836791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:03.836791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:03.836791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:03.836791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:03.839259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 381/760] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:03.840333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.880676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 380/761] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:03.882325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a4178-683d-42f2-a681-a00941dba48f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:03.935912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:03.986610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16e440-562f-450d-8a69-2359bb212365 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:03.989199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a4178-683d-42f2-a681-a00941dba48f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.026726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.027191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.028295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.028769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.029056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.029212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.029925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.030783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.032298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:04.033054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.035720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.041914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.044553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.058269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 381/762] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:04.060175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:04.070138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:04.071911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:04.074059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 382/763] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:04.075048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.156842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:04.203336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b72b7-a1c8-425b-857d-819f9bf4a801 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.206090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:04.212989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a23b2-316e-4de1-af16-2ebd9928eccb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.239695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.240613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.240967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.241318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.242581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.244067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.251499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.252525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.252980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.253425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:04.257914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:04.258840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:40:04.259729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:04.264192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.264311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 382/764] 10.222.0.86 () {56 vars in 1129 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:04.265181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.266104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:04.281792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:04.282682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:04.283717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:04.286426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 383/765] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:04.287385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.369291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39eebc-f6b1-4282-91fb-d5d9a9278385 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.396075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abd91-253a-4e3e-ab6e-d1a6ca41f695 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.404634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.405691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.405691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.405980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.406849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:04.407278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.408782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.420691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:04.422726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 383/766] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:04.423608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.439231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.440346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.440795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.441237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.442145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:04.442647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.444397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.449946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 384/767] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:04.450966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.457869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:04.534747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507ea7b-f85e-4f9c-a613-6a99164ce140 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.572817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.573479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.573766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.574038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.574824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:04.575176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.576613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.578545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0af78-c18e-442c-bdcc-a848f8850542 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.590132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 384/768] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:04.595107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.635910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.637473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.638207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.638653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.639733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=d3d4323286624d928d94a6486d299eea) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:04.640246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.642949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.652212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 385/769] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/role_assignments?user.id=d3d4323286624d928d94a6486d299eea&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:04.653314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.758897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:04.767313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:04.786849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac5bcb-b6f9-4b6d-8acb-a3b1ba77b523 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.857547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.857868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.858162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.858442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.859319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:04.859636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.861274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.865578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55ff0d-1d44-458d-8fec-7a67d65f25fe None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:04.873806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 385/770] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:04.875267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:04.917089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46813e9a-80e0-42ee-9fb4-8b744f05bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:04.918256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46813e9a-80e0-42ee-9fb4-8b744f05bad2 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:04.918681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46813e9a-80e0-42ee-9fb4-8b744f05bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:04.919232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46813e9a-80e0-42ee-9fb4-8b744f05bad2 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/e06439f92e494fb4a68ffee7bdb0ffc5/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:04.921505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46813e9a-80e0-42ee-9fb4-8b744f05bad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=e06439f92e494fb4a68ffee7bdb0ffc5, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:04.923409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46813e9a-80e0-42ee-9fb4-8b744f05bad2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:04.947325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 386/771] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:04 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/e06439f92e494fb4a68ffee7bdb0ffc5/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:40:05.000148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286a3da-e471-4360-a5cc-a854d3366bee None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:05.056235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:05.057275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.057760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.058237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.059270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] RBAC: Adding query filter params (interface=public, service_id=8b02322762b74c0595f5c95ccb660b84, region_id=RegionOne) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:05.059820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:05.061397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:05.072255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8a039-a57b-4e73-8d2a-8d58e0106bf8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.072778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8a039-a57b-4e73-8d2a-8d58e0106bf8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.072778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8a039-a57b-4e73-8d2a-8d58e0106bf8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.074420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 387/772] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:05.080000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.080269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.080552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.092639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 386/773] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:04 2022] GET /identity/v3/endpoints?service_id=8b02322762b74c0595f5c95ccb660b84&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:05.095332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:05.121768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:05.123393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:05.141029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:05.200212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 388/774] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:05.234319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:05.286361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:05.287771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcd126-2830-4c7d-9541-2d35ff8f00f6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:05.326461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:05.327227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.327469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.327999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.329169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:05.330342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:05.343086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 387/775] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:05 2022] GET /identity/v3/services => generated 772 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:05.395721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677dc0d6-4698-44b0-b789-d8c98f3ed002 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:05.450032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:05.450860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.451164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.451442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.452112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:05.453540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-abd41ba0-d9b2-4d78-8f44-6bd39f6021c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:05.474032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:05.475162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:05.476735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 389/776] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:05 2022] GET /identity/v3/services/network => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:05.484221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:05.591952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:05.615621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13f047-1650-4cb4-86ab-8f9d4407ec4a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:05.657826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d0f64-4991-4350-bc81-fefa08cd0a88 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.658132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d0f64-4991-4350-bc81-fefa08cd0a88 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.658392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d0f64-4991-4350-bc81-fefa08cd0a88 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.659208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:05.659808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 390/777] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:05.660192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.660621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.661062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.661931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] RBAC: Adding query filter params (name=network) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:05.662388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:05.664677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:05.667249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.667525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.667801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.670941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 388/778] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:05.676389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:05.699491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:05.700701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:05.834680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 391/779] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:05.856239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:05.865940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c509e24-db69-456e-a01f-f783677444fe None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:05.888162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:05.956923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:05.957606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:05.957891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:05.958168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:05.958895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] RBAC: Adding query filter params (type=network) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:05.959234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:05.960717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:05.966944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce84b14-57b7-422b-af76-1b01cadcb831 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:05.979307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 389/780] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:05.985149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:06.004668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:06.032235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.033162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.033383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.033662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.046647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:06.055253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:06.056255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:06.057434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 392/781] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:06.075152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:06.152199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d196a-5f97-4706-b8e7-810fa82d563b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:06.212673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.213834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.214353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.214847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.216013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] RBAC: Adding query filter params (interface=public, service_id=1b17349edc144a159b022af2a6adb47d, region_id=RegionOne) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:06.216550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:06.223600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:06.233355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 390/782] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:05 2022] GET /identity/v3/endpoints?service_id=1b17349edc144a159b022af2a6adb47d&interface=public®ion_id=RegionOne => generated 194 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:06.238889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:06.267039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:06.277465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65341809-84f2-4204-a064-529148ad41d4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:06.312447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.313116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.313401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.313672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.314423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:06.314730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:06.316154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:06.330059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 393/783] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:06.331303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:06.435254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82334b0-bf04-4f9e-b0e0-b2f4c345c88b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:06.463929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:06.474691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.476429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.476909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.477365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.490104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:06.491632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:06.504449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b8513-0b02-49ee-b2a9-12974fac2adc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:06.509968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:06.510894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:06.511864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:06.511864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:06.511864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:06.511864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:06.516152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 394/784] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:06.517425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:06.527854 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:06.548608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.549329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.549616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.549891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.550682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:06.552211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:06.558978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 391/785] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:06 2022] GET /identity/v3/services => generated 772 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:06.676118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f525f9f-c575-4bbf-80bb-6ca6612ee482 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:06.714941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.715880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.716339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.716766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.717657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:06.718129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:06.719695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:06.733805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 395/786] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:06.739686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:06.871803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad97c-47cc-450f-8d36-cfe7e6e92a58 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:06.906895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:06.907781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:06.908059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:06.908333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:06.911227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:06.917843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:06.919596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:06.935101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:06.936112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:40:06.937084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:06.940968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 392/787] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:06.942365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.018136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:07.056599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd723c11-150e-4673-8aea-3f8ef5666dc4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.094169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.094904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.095161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.095438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.096398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:07.096711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:07.098141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e253b-274d-4b50-8bab-1e9496ba1be6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:07.121779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 396/788] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Jul 20 21:40:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:07.123211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.202253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e60be-4b76-4344-915f-1acaf35642d3 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.202585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e60be-4b76-4344-915f-1acaf35642d3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.202810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e60be-4b76-4344-915f-1acaf35642d3 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.204521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 397/789] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:07.210729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.211021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.211505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.229440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:07.230760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:07.255330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f0344-eabd-48a1-8984-ae6989bb6cd9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.297122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.297844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.298131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.298407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.312403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:07.313890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:07.334071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:07.336731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:07.338745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:07.338745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:07.338745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:07.338745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:07.338745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 393/790] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:07.348081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.353527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 398/791] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:07.378014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.441683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:07.473239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:07.537586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b0d4f-a72e-4dd0-b9a7-896448fec64f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.556026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443c620-6368-452d-9f94-b1ff4df0baf7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.591714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.592381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.592693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.593035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.593871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:07.594183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:07.595605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f672d4ad-d682-46a8-b308-18da0958302f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:07.604801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.605752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.606029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.606299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.609137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 394/792] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:07.610731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a1b5-4587-4752-b922-6a49a0d21181 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.611020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a1b5-4587-4752-b922-6a49a0d21181 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.611322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a1b5-4587-4752-b922-6a49a0d21181 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.612823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 395/793] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:07.620001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.620001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.620001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.622857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:07.631114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:07.632719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:07.633625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 399/794] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:07.634354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.640920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:07.642377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:07.723344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 396/795] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:07.724714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.743590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f7c2a-0b31-44e7-88fe-3a1f073d68cd None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.758954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:07.781772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.782569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.782845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.783136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.786652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:07.792569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:07.794041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:07.811916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:07.812861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:07.813789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:07.818493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 400/796] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:07.820184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.857194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9578a2-7c7a-4ec0-ba31-64df48da2cd4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.899591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.900267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.900548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.900820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:07.901693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:07.901998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:07.903409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:07.912559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 397/797] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:07.913503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:07.946473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb88c7f-4208-4050-82c0-6737ee869659 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:07.988095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:07.989208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:07.989323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:07.989597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.001563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:08.010631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:08.011651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:08.012522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 401/798] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:08.013269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.023967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.029086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14059c36-bd11-4d09-82fd-edca69f98025 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.056579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.068543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.069301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.070340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.070574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.071593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:08.071987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.073611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74275a-b9c5-4fe6-8215-50f8e96927df None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.099188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 398/799] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:08.099188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa1dba-f3eb-4336-b4a1-a0b0c1336d15 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.099188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa1dba-f3eb-4336-b4a1-a0b0c1336d15 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.099188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa1dba-f3eb-4336-b4a1-a0b0c1336d15 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.099188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 399/800] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:08.099636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa1dba-f3eb-4336-b4a1-a0b0c1336d15 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.133504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a05b28-3c79-4c5e-b4b6-f066571b5144 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.169982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.170889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.171213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.171488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.183697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.185162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-05b8efec-34e5-4d6d-b2c6-8a9c243f1e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:08.204080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:08.205041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:08.206552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:08.206552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:08.206552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:08.206552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:08.206552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 402/801] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:08.206843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.207099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.207387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.225524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:08.226290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa1dba-f3eb-4336-b4a1-a0b0c1336d15 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.227411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:08.268111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.268782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.269058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.269329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.270116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:08.270381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.271774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.284953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 400/802] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:08.284953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.308337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.309322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 403/803] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:08.310251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.329252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.400692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec512cc-adbc-4a2d-a868-09e48f2e1b83 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.414870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668c7dc-d61b-4383-b6b9-e5f4c6dc5439 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.439563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.440688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.440974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.441251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/6952f7293b2f46f28941b48429dd2e73/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.443470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=6952f7293b2f46f28941b48429dd2e73, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.444949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.451613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.452280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.452597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.452829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.453631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:08.454020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.455433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.467301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 401/804] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:08 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/6952f7293b2f46f28941b48429dd2e73/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:40:08.467301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.476364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 404/805] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:08.477919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.557296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.581254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a281c53-82fc-4132-8360-4693e7f158bd None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.593633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.625250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.626059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.626339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.626610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.629429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c454c9-a831-43cc-9836-5bf91f04562a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.639324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.640808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:08.659363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:08.660359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:08.661269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:08.661269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:08.661269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:08.661269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:08.661269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 402/806] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:08.662020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.672708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.673850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.674137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.674421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.675124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.676519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:08.691110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:08.692106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:08.693560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 405/807] 10.222.0.86 () {56 vars in 1129 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:08.694432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.793429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89165648-1bf5-4a70-a75c-086717cc4255 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.827281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cd40d-3575-48e6-ab04-bc2e0a50a598 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:08.831190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.831458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.831685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.831965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.834145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.841253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.843321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:08.861494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:08.862719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:08.863921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:08.865896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 403/808] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:08.867326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.878912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:08.880294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:08.880725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:08.881601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:08.883322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:08.883778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:08.885576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:08.898174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:08.905264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 406/809] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:08.906521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:08.978483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6b3b1-4edb-4791-b77e-c0936274ab87 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.015537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.016348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.016752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.017127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.017953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:09.018361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.019854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.028064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 404/810] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:09.029682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.042146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc95d82-d2a3-44a5-b28c-06609d578324 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.080794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.081709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.082034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.082416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.083404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:09.083826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.085709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.106557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 407/811] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Jul 20 21:40:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:09.107640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.113467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.171209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcdb310-1453-4145-aeb4-64b4ad42bab4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.176066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.218150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.218951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.219264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.219541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.222338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874898e-fb5d-4a0b-bf0c-1d5feaa1b477 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.228407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.229885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:09.245403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:09.246429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:09.247436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 405/812] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:09.248055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.267926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.268609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.268893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.269170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.269890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:09.270201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.271630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d27586-c70a-4723-bc7f-b3919d1c0e70 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.278230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 408/813] 10.222.0.86 () {56 vars in 1124 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:09.279531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eb1f3-dfdb-4ba7-84fb-b32776b91bf0 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.279821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eb1f3-dfdb-4ba7-84fb-b32776b91bf0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.280098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eb1f3-dfdb-4ba7-84fb-b32776b91bf0 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.283163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 409/814] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:09.283163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eb1f3-dfdb-4ba7-84fb-b32776b91bf0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.365604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5edcb-5159-46ea-a275-88e8cad79397 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.384186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.396483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eb1f3-dfdb-4ba7-84fb-b32776b91bf0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.403519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.404322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.404604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.404880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.416340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.417796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.422040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.435282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:09.435534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:09.436423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:09.437351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:09.437351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:09.437351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:09.437351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:09.437588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.437588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.437588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.437588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:09.437588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.439052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311b557-21e3-4af6-803a-6e9e1ca31927 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.441348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 406/815] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:09.442169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.459388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 410/816] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:09.461529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.461807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.462081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.481643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:09.482841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:09.559282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae1f0d-6e28-4b1a-9616-88550271977d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.559479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 411/817] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:09.559479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.598162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.598855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.599168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.599447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.600188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.601694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.629126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 407/818] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:09.630880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.685050 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c710a-51da-47c5-9e2c-1f32fa3c81e8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.691505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.702878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.742107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.743294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.743733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.744299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.745202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:09.745676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.747628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.762407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 412/819] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:09.763635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.803645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237a6d2-88b2-45b5-8f3d-21dd3ddc7fbf None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.842441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.843293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.843681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.844222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.882713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7abc6c-fb5e-464f-8371-27447ac5f0bb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:09.921785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:09.922572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:09.922866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:09.923161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] PATH_INFO: `/v3/services/network` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:09.924411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.925203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.925323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:09.926690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:09.938323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:09.939539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:09.940831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 413/820] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:09.941663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:09.965936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:09.984986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.051364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 408/821] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:10.052691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.063995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684c83a-7a5a-43b0-a55f-15a56a9f8cfe None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.101342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.102157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.102463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.102901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.109926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.111690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:10.126649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:10.127656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 414/822] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:10.129173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.169731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb697e-0649-4906-8e89-1f1dca979a70 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.198640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.209446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.211119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.211627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.212095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.213103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] RBAC: Adding query filter params (name=network) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:10.213605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.215319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.221174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 409/823] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:10.222238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.235248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.250892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da92b-ccce-4760-89b1-95ff6abeba3c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.306008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.307268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.307804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.308280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.323806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.326390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.334883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc35857-8785-4a08-98d1-e0fb4d132181 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:10.350115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:10.352455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:10.353376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:10.353376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:10.353376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:10.353376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:10.355920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 415/824] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:10.357074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.487556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e6449-e847-4edb-b43b-338f9f500a75 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.500867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.502160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.503172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.503682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.504917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:10.505652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.509114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.527303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.527586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 410/825] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:10.528474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.528710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.529232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.529683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.530592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] RBAC: Adding query filter params (type=network) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:10.531395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.534039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.542396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 416/826] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:10.543685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.555371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.597267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.658372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446c451-c6d5-459c-85dd-e471e9237b90 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.680068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5b04a-7dec-4f61-a3f7-5e92d5cc690e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.694313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.695342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.695537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.695880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.696669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:10.697041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.698487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.709939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 411/827] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:10.710732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.720814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf0cb6-a7c3-45c6-b3d0-2722a2d34438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.721784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf0cb6-a7c3-45c6-b3d0-2722a2d34438 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.722134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf0cb6-a7c3-45c6-b3d0-2722a2d34438 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.722483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf0cb6-a7c3-45c6-b3d0-2722a2d34438 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/d3d4323286624d928d94a6486d299eea/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.738871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf0cb6-a7c3-45c6-b3d0-2722a2d34438 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=d3d4323286624d928d94a6486d299eea, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.740364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf0cb6-a7c3-45c6-b3d0-2722a2d34438 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.755164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 417/828] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:10 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/d3d4323286624d928d94a6486d299eea/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:40:10.756793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794abe2-2b96-4c14-9635-1256411051a5 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.757257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794abe2-2b96-4c14-9635-1256411051a5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.757555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794abe2-2b96-4c14-9635-1256411051a5 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.763421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 418/829] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:10.763421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794abe2-2b96-4c14-9635-1256411051a5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:10.829419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.835911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1e8f5-991b-469f-b18d-98f47fd89be4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.847594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:10.874306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837dde72-de36-4732-bec1-681e22965129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.874985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dde72-de36-4732-bec1-681e22965129 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.875300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dde72-de36-4732-bec1-681e22965129 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.875577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837dde72-de36-4732-bec1-681e22965129 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.876337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dde72-de36-4732-bec1-681e22965129 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.877729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837dde72-de36-4732-bec1-681e22965129 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:10.888967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794abe2-2b96-4c14-9635-1256411051a5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:10.898784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 412/830] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:10.900655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.901200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.901634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.923980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:10.925636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:10.967844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:10.968670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:10.968950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:10.969222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:10.979806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:10.981870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:11.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:11.004386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:11.005503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 419/831] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:11.012355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.017589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 413/832] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:11.042135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.162978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6146730-ef84-41d0-9bd2-b90ff74ffe1d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:11.177511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:11.200166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:11.201063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.201387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:11.201666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.202117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.203091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:11.203552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:11.205152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:11.211524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c80579c-86f6-42e5-81ed-95a795bd3f3c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:11.219344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 420/833] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:11.229969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.274321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:11.275033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.275384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.275661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.291438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:11.300800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:11.301778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:11.302642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 414/834] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:11.314321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.383677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef5d21-1b8d-457f-b0b8-98deeb93d8a0 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:11.431545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:11.432961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.433446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.433946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.435568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=e06439f92e494fb4a68ffee7bdb0ffc5) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:11.436108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:11.438313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:11.448800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 421/835] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/role_assignments?user.id=e06439f92e494fb4a68ffee7bdb0ffc5&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:11.450839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d017183-d9d8-4667-8853-6eca4378e61b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:11.492940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:11.493615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.493893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.494165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.494933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:11.495265 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:11.496674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:11.507455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 415/836] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:11.512788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.563580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:11.630814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e3d14-09d8-4f32-ad85-c5fccb90c2a6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:11.666985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:11.667819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.668095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.668368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.682634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:11.684163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:11.702885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:11.703881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:11.704822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:11.704822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:11.704822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:11.704822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:11.704822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 422/837] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:11.712201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.733633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:11.821652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee281d36-0c69-4093-bd2a-b8a00418c625 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:11.858279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:11.858940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.859252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.859527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.860244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:11.860809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:11.862151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d50ae8-3ed5-42cb-ab9f-676e75fa8b1c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:11.874870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 416/838] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:11.884117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:11.932828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b9d94-8936-423d-8ffa-5673e463f082 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.933141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b9d94-8936-423d-8ffa-5673e463f082 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.933381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b9d94-8936-423d-8ffa-5673e463f082 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.934756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 417/839] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:11.940731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:11.941018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:11.941297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:11.962916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:11.964139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:11.995704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:12.011762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4482a-377d-442c-ae62-f20565c93315 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.043083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 418/840] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:12.058993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.060897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.062988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.063536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.064039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.070882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:12.075296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.077652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:12.099440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:12.100495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 21:40:12.101840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:12.105347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 423/841] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:12.111914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.187989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba095a5-bd7b-4f8c-a294-73cda90d5556 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.239336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.240281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.240661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.241034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.241833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.242370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07da3b-56ef-4dea-b866-843bce8e2a18 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.243546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:12.256562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:12.258305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:12.259357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:12.261653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 419/842] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:12.267028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.288489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.289767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.290230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.290676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.291745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:12.292716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.294585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.322367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 424/843] 10.222.0.86 () {56 vars in 1135 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:12.323416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.332348 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:12.350156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:12.384335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a50d-8659-42b3-bcbf-1861099b6307 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.419284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.420276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.420755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.421246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.422139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:12.422648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.424216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.430028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 420/844] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:12.434459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a943-de50-4461-b6d1-ad894497b578 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.436195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.468353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.469009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.469295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.469576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.541697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707ef98-4bbe-404a-9455-63b10f5cd277 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.542362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.543908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.552684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:12.579763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.580793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.581269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.581737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.582686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:12.583238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.584792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.587966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:12.599461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 421/845] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:12.662280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 425/846] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:12.668608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.791422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ad510-94ee-43eb-8960-e264a0d741bf None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.827437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:12.828237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:12.828522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:12.828802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:12.840016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:12.841399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:12.859403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:12.860581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:12.861508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:12.861508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:12.861508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:12.861508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:12.861508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 422/847] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:12.867337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:12.973847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2f9cc-9ab4-4773-8fc1-8ff45f7309eb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:12.982581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.012754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.013421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.013709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.013989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.014720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:13.015029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:13.016435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524c4c1-34ec-455e-99f4-06b753928f8c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:13.029132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 426/848] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:13.034687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.069691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20fb80-4447-44e3-887b-c7805482c918 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.070199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20fb80-4447-44e3-887b-c7805482c918 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.070483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20fb80-4447-44e3-887b-c7805482c918 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.072306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 427/849] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:13.077596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.078147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.078574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.101390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.103581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:13.104885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:13.161586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d7166-4faf-4985-858f-53abd4889c49 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:13.197882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 428/850] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:13.203724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.204771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.205258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.205739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.213656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:13.215388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:13.222510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:13.232790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:13.233723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:13.234624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 423/851] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:13.240904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.343432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.354581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f87eba-9293-4cd7-9f2e-159cc2b7246f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:13.366642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56726187-1ccf-43d5-b742-504f25a43a0e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:13.380849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.394373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.395209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.395494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.395776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.405741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.406685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.407212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.407711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.408619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:13.408919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:13.409427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:13.411079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:13.417450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:13.418431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:40:13.419331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:13.423401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 429/852] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:13.424839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.428187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 424/853] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:13.434644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.551624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f67cae-ea63-4164-a6f2-8c6b96bdafde None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:13.556402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bde929d-85ac-4c2e-9a4f-ae40ccf4ef96 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:13.591569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.592237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.592520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.592796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.593635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=6952f7293b2f46f28941b48429dd2e73) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:13.593951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:13.595417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:13.606062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.607113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.607471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 425/854] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/role_assignments?user.id=6952f7293b2f46f28941b48429dd2e73&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:13.607820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.608314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.609318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:13.609835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:13.611966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686e76-23a4-40df-9380-a7808461bac5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:13.622710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 430/855] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:13.628272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.669644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.733190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ec86a-3c8e-4745-97c5-3d2fd456e28b None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.733520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ec86a-3c8e-4745-97c5-3d2fd456e28b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.733747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ec86a-3c8e-4745-97c5-3d2fd456e28b None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.735202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 431/856] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:13.741873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.742315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.742795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.759733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5930a3d-e53c-43db-bdc5-b1e0b35db7fc None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:13.762131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:13.763814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:13.779897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.812116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:13.813063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:13.813204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:13.813468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:13.826906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:13.828427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:13.853328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:13.854287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:13.855263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:13.855263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:13.855263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:13.855263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:13.855263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 426/857] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:13.858287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 432/858] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:13.860203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.880924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:13.964758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:13.978325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0649a-cf6e-431b-a87f-38642384d29a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.017639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.018392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.018747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.019114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.019918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:14.020336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.021811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.026735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0beb19-5f7c-4386-8d61-2edd4fa5f214 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.040406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 427/859] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:14.041596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.163873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55092b5a-f2a1-4a78-bbc9-c5dc4c918e4f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.210015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.211726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.212471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.212760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.220423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.221367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.221744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.222115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.222379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.222981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.223855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.224578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:14.238431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:14.239632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:14.240693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:14.247095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:14.249216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:14.250742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:14.251818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 433/860] 10.222.0.86 () {56 vars in 1132 bytes} [Wed Jul 20 21:40:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:14.251962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 428/861] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:40:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:14.252103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.254541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.266721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:14.273284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:14.371445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad25d7-5020-4d64-abb8-c4aaa7fc49a9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.385137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebd4a7-0687-4e97-9cb0-92996053dc3a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.409080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.410046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.410157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.410532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.411338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:14.411715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.413313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.423167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 429/862] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:14.425158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.437383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.438079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.438724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.438724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.439508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:14.439821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.441261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.465608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 434/863] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Jul 20 21:40:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:14.466525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.520330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:14.541491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:14.564254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8844fa-46d2-4d2e-9bdf-f55fe185e534 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.579365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1560e7-781c-465a-8d6a-5fdd817d0a06 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.603910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.604762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.605149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.605524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.606337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:14.606748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.608552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949d337-ba1a-4110-9676-db8c3b384903 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.619092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 430/864] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:14.635111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:14.635807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.636093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.636373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.708047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181506-5e2b-4a3c-9060-6838becf0ace None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.708348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181506-5e2b-4a3c-9060-6838becf0ace None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.708541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09181506-5e2b-4a3c-9060-6838becf0ace None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.711191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 431/865] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:14.715714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:14.715985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:14.716241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:14.716517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:14.718022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:14.734050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:14.735330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:14.799307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:14.820916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 432/866] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:14.839634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 435/867] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 21:40:14.842473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.847770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:14.920596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:14.967825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79360d-9450-40d8-a980-37873c32b6e7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:14.976894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf011e-e539-4777-b651-ed540691c86a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:15.004974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:15.005779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.006155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.006525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.007745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:15.009208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:15.020024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:15.020891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.021106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.021438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.024834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 433/868] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:14 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:15.035586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:15.037203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:15.059982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:15.060976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:15.061896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:15.061896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:15.061896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:15.061896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:15.061896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 436/869] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:15.068524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:15.211137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdcf91-a652-4408-a48a-a7e5f351a0d8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:15.236833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:15.254771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:15.255470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.255750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.256024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.256753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:15.257086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:15.258451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:15.270978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 434/870] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:15.276652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:15.302255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:15.381434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766cd73-c7b3-4fbc-8d08-82c76f2afdee None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:15.417951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:15.419181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.419673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.420151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.427726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:15.429475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:15.443903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:15.445027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:15.445948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:15.448518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 437/871] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:15.453951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:15.561855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54224ab6-4d20-4732-92ec-4a2856acf767 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:15.599582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:15.601739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.602252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.602731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.603761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:15.604293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:15.605913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab48b8b-9b21-4901-82e1-58e9af5cc502 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:15.619789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 435/872] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:15.628374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:15.717330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:15.736675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8c6a4-f1a7-4e57-813d-efb007f3f778 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:15.771371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:15.772042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.772326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.772633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.773473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=d3d4323286624d928d94a6486d299eea) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:15.773786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:15.775256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ace764-7dfa-43a8-99ed-0078fc3efbcd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:15.781825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 438/873] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:15 2022] GET /identity/v3/role_assignments?user.id=d3d4323286624d928d94a6486d299eea&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:15.808014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dff7c-376c-4f9f-a1bb-40adf871f584 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.808405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dff7c-376c-4f9f-a1bb-40adf871f584 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.808589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279dff7c-376c-4f9f-a1bb-40adf871f584 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.810316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 436/874] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:15.816245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:15.816527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:15.816733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:15.835193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:15.836445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:15.916559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 439/875] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:15.939353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dff7c-376c-4f9f-a1bb-40adf871f584 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:15.993575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:16.051877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279dff7c-376c-4f9f-a1bb-40adf871f584 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:16.098135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:16.099237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:16.099683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:16.100121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:16.101634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:16.103205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:16.115989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:16.117567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:16.118667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:16.121275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 437/876] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:16.126915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:16.208268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:16.240427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1f1d6-1133-4aae-b19a-dbdd0c5a8017 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:16.276051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:16.276728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:16.277014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:16.277294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:16.278013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:16.278327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:16.279727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c415ca2-ab53-434b-87b6-ddfe46c055cf None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:16.284857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 440/877] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:16.291047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:16.409061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7169d-c17b-4aed-99e1-cc0673fb8a76 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:16.416997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:16.446152 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:16.446979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:16.447290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:16.447568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:16.448333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:16.448647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:16.450008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5652016-46de-46a4-959e-d7dac82ce997 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:16.455180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 438/878] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:16 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:16.806767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae0a62-b164-455b-8239-8a82c28e01b0 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:16.807518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae0a62-b164-455b-8239-8a82c28e01b0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:16.807518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae0a62-b164-455b-8239-8a82c28e01b0 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:16.808967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 441/879] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:16.814237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373ec7-2a49-4b9a-851f-2a3d4d3eb0fa None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:16.814730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373ec7-2a49-4b9a-851f-2a3d4d3eb0fa None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:16.814901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4373ec7-2a49-4b9a-851f-2a3d4d3eb0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:16.833227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a4373ec7-2a49-4b9a-851f-2a3d4d3eb0fa None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:16.834407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a4373ec7-2a49-4b9a-851f-2a3d4d3eb0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:16.906395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 439/880] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:16.929519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae0a62-b164-455b-8239-8a82c28e01b0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:17.036587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae0a62-b164-455b-8239-8a82c28e01b0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:17.080822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58312626-cbb4-4f6d-8aca-de06c5a8ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:17.081793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58312626-cbb4-4f6d-8aca-de06c5a8ce56 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:17.082296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58312626-cbb4-4f6d-8aca-de06c5a8ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:17.082785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58312626-cbb4-4f6d-8aca-de06c5a8ce56 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:17.083769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58312626-cbb4-4f6d-8aca-de06c5a8ce56 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:17.085675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58312626-cbb4-4f6d-8aca-de06c5a8ce56 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:17.101087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:17.110702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 442/881] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:16 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:17.303195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863efcee-baeb-43c2-ab6e-23e41e65c376 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:17.303519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863efcee-baeb-43c2-ab6e-23e41e65c376 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:17.303813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863efcee-baeb-43c2-ab6e-23e41e65c376 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:17.305837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 440/882] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:17.313438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:17.313438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:17.313438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:17.333235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:17.334781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:17.415164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 443/883] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:17.433391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863efcee-baeb-43c2-ab6e-23e41e65c376 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:17.543698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:17.547916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863efcee-baeb-43c2-ab6e-23e41e65c376 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:17.584677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:17.586451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:17.586741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:17.587018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:17.587717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:17.589085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:17.602777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:17.603862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:17.605153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 441/884] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:17.612648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:17.718370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d855f-9c12-4161-bd0d-3cef4b1ac463 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:17.746961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:17.756427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:17.758465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:17.758922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:17.759381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:17.760236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:17.760709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:17.762234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:17.767752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 444/885] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:17.773498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:17.885204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4f95-dd9d-47b5-a923-4ef0346b785e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:17.920419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:17.921090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:17.921372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:17.921645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:17.922360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:17.922670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:17.924071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c646b02-96b5-4666-a73f-886dfccbe289 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:17.930715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 442/886] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:17.936656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.030156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ec02e-fd35-45e6-8202-4c1e2c1d27f3 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.030662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ec02e-fd35-45e6-8202-4c1e2c1d27f3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.031552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ec02e-fd35-45e6-8202-4c1e2c1d27f3 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.033781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 443/887] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:18.039647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.040216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.040709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.042048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad135e-0486-43f1-9f2f-d16a47ff61c2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.059207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:18.060782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:18.086987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.087918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.088366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.088884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.089973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] RBAC: Adding query filter params (interface=public, service_id=f9493a4fe4c543dd8e021b4efc8d4d45, region_id=RegionOne) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:18.090573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:18.091978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:18.096830 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:18.103509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 445/888] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:17 2022] GET /identity/v3/endpoints?service_id=f9493a4fe4c543dd8e021b4efc8d4d45&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:18.109507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.159228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 444/889] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:18.177851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:18.183349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.229934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968d1be-267c-4903-b898-3002c7b99e79 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.267003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.267985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.268418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.268835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.269662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:18.271249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:18.277836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 446/890] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:18 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:40:18.300451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1c288-d5f2-435d-941b-8bef3097aaa4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.342620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.343892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.344393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.344861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.359196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:18.368488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:18.369518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:18.370365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:18.373004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 445/891] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Jul 20 21:40:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:18.380294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.497866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5749c-ba48-451d-a67d-e93ad81bd259 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.515932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:18.535497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.536496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.536930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.537340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.538832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:18.539315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:18.540922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d1fc9-7b38-458f-8095-0d245adadff3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:18.549661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 447/892] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:40:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:18.555574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.576153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:18.643135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c28594-36f9-4359-8234-4583efc528c8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.643791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c28594-36f9-4359-8234-4583efc528c8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.644709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c28594-36f9-4359-8234-4583efc528c8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.646570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 448/893] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:18.652931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.653332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.653786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.668338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52573e24-bb29-46ae-bd26-93f1fa692134 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.671719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:18.673005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:18.707901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.708974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.709434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.709883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.725012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:18.727886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:18.751800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:18.752791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:18.753719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:18.753719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:18.753719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:18.753719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:18.756705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 446/894] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:18.767110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.767296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 449/895] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:18.792494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.862612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:18.897719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcb0f9-c475-4901-8464-c657b851a739 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.923501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed504b-b593-4987-8ec0-715d00de4d2e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:18.937265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:18.941831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.942506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.942944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.943258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.943998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:18.945296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:18.946995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:18.963245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 447/896] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:18.967229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:18.968072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:18.968487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:18.968673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:18.969035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:18.969899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:18.971471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:18.988941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 450/897] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:18 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:40:19.107946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65666461-fefa-4158-97da-cb4a36d81c7d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:19.140539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:19.153424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:19.154248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.154526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.154797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.163664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:19.165104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-ea1ee7ce-3341-4051-8920-ac9d55e5fb82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:19.181102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:19.182140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:19.183157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:19.185715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 448/898] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:40:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:19.188322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.296034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6772d400-40b7-41cd-8cfc-a395d077844b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:19.332090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:19.332684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.332962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.333234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.334130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:19.334438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:19.336423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:19.348002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10cf22-930e-4872-8f7c-d2b184d55ef9 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.348291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10cf22-930e-4872-8f7c-d2b184d55ef9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.348556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10cf22-930e-4872-8f7c-d2b184d55ef9 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.350194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 449/899] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:19.355816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 451/900] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:19.356071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.356172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.356432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.364085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.379069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:19.380403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:19.394590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:19.459341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 450/901] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:19.480315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.485718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7a128-b6fe-44f0-a5e6-39e9459bff30 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:19.535105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:19.536165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.536665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.537137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.598707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543dd700-c8bc-4627-98a7-18601138b6e7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:19.609002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:19.610976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:19.617019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:19.637447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:19.638266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.638547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.638822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.639671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:19.640881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:19.642995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:19.655196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:19.656206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:19.657069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:19.661491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 451/902] 10.222.0.86 () {56 vars in 1132 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:19.662343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.724675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 452/903] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:19.734435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.775204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cc392-3e47-4673-8987-ef43ebfc6345 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:19.811336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:19.812624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.812923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.813205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.813925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:19.814239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:19.816309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:19.827249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 452/904] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:19.829783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.861453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f06f03-3ac6-4dd2-9ef1-808e06b8bfed None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:19.892006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:19.895417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:19.902454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:19.903602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:19.904114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:19.904594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:19.916006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:19.917726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:19.937938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:19.939572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:19.940499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:19.940499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:19.940499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:19.940499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:19.942554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 453/905] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:19.949349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:19.962889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a293e04-753d-4799-baf6-4820c18d02c0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.002068 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.003124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.003637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.004124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.005150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:20.005678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.007971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.014279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 453/906] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:20.019645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.059681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84aea8-c48f-4fba-8773-e6e5b7b55800 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.095679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.096349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.096634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.096914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.097650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:20.097962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.099401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.112702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 454/907] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:20.119491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.150662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb69b6e-2956-4f25-ac01-204af63a932d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.163165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:20.173484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:20.200095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.201093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.201582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.202087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.203241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] RBAC: Adding query filter params (interface=public, service_id=09c93120e4a745e4a2fcbfe12e3ad0b6, region_id=RegionOne) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:20.203855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.205450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.212013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 454/908] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/endpoints?service_id=09c93120e4a745e4a2fcbfe12e3ad0b6&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:20.218048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.236385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576bbc1-c1a1-4dda-bd70-579d70719f4a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.273833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.274668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.274955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.275266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.283024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.284515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:20.300535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:20.301524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:20.302917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:20.307087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 455/909] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:20.307979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.343332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68de9ff-3e22-4e30-ba9e-81fdef18cd08 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.383457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3cce88-df1b-4b36-84c0-15cb8756d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.384498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cce88-df1b-4b36-84c0-15cb8756d652 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.384988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cce88-df1b-4b36-84c0-15cb8756d652 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.385492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3cce88-df1b-4b36-84c0-15cb8756d652 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.386415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cce88-df1b-4b36-84c0-15cb8756d652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.388144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3cce88-df1b-4b36-84c0-15cb8756d652 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.403162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 455/910] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:40:20.412064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:20.417549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710819f-f10b-4652-8f7b-f45789b5633f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.458599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.459320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.459607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.459913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.460727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:20.461038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.462625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.475120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b32d0-7b93-499b-af0b-38eaa89c5493 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.476282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b32d0-7b93-499b-af0b-38eaa89c5493 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.476785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b32d0-7b93-499b-af0b-38eaa89c5493 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.477033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 456/911] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:20.479006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 456/912] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:20.482867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.484279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.484802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.485333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.505451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:20.507477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:20.542111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:20.590381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 457/913] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:20.609233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7082e61-d1d1-40d4-9242-fd854d25fc84 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.615438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.645101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.646006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.646437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.646926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.648594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] RBAC: Adding query filter params (role.id=3704a72fb24e4d2fb163fb0213f53e89, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=d3d4323286624d928d94a6486d299eea) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:20.649052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.650636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.663657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 457/914] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/role_assignments?user.id=d3d4323286624d928d94a6486d299eea&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=3704a72fb24e4d2fb163fb0213f53e89 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:20.762920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cf90c-46f1-4061-81c1-c7f63a6e6a54 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.801487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.802302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.802585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.802863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.804077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.805603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:20.818677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:20.820309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:20.822355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 458/915] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:20.826394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:20.855226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:20.935759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43098a2a-4d46-4223-9beb-e3c612ac0ce1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:20.974535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:20.975221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:20.976091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:20.976993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:20.977719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:20.978033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:20.979437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:20.985008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 458/916] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:20.990620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:21.023928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:21.104832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc490e05-fed8-4936-b643-22393ece33e3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:21.141570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:21.142239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.142524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.142802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.143556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:21.143869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:21.145239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ac6d2-f7bb-4be4-97da-e3cd569c547e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:21.151140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 459/917] 10.222.0.86 () {56 vars in 1114 bytes} [Wed Jul 20 21:40:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:21.156888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:21.256247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:21.275492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462687c-31c8-4585-876f-06501f96c7b4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:21.279270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815017c-7f16-4221-aa67-ac45703e527a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.279560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815017c-7f16-4221-aa67-ac45703e527a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.279837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815017c-7f16-4221-aa67-ac45703e527a None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.281351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 460/918] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:21.287083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.287872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.288844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.310188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:21.312127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:21.315027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:21.315716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.315996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.316266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.317013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:21.318381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:21.341891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 459/919] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:21.395963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 461/920] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:21.420856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:21.542755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27303966-ed28-4464-ad6f-1a4d93dde4dd None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:21.579142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:21.580339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.580833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.581331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.582245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:21.583942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:21.597063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:21.598079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:21.598937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:21.602650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 460/921] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:21.608547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:21.652905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:21.716216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14e133-3669-4f46-8c48-b2a67c95b0a9 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:21.753167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:21.753924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.754210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.754507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.755340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:21.755650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:21.757138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:21.762609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 462/922] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:21.768491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:21.782630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:21.877354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a57e0-dcd4-41ad-ba5b-c3207ebe001e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:21.912751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:21.914943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:21.915479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:21.915940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:21.916863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:21.917353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:21.919099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:21.927479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 461/923] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:21.934682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:22.025527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:22.043742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37c061-b2f0-4ae5-b221-8b58d5021e1d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:22.080184 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:22.083083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.083083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.083083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.083083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] RBAC: Adding query filter params (interface=public, service_id=5458d0bb435c4d04840048d8a6b77215, region_id=RegionOne) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:22.083083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:22.083584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4543504-f64b-4bf7-b3b3-89428dfc32cf None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:22.090361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 463/924] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:21 2022] GET /identity/v3/endpoints?service_id=5458d0bb435c4d04840048d8a6b77215&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:22.095952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:22.169439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:22.204052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d605b1-8dd5-4010-83a3-0691a2b5d0c9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:22.239337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057e85f-bfbd-4608-aced-e5edc9ca17bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:22.240008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e85f-bfbd-4608-aced-e5edc9ca17bd None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.240292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e85f-bfbd-4608-aced-e5edc9ca17bd None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.240569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e85f-bfbd-4608-aced-e5edc9ca17bd None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.241311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057e85f-bfbd-4608-aced-e5edc9ca17bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:22.242869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057e85f-bfbd-4608-aced-e5edc9ca17bd None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:22.249419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 462/925] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:22 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:40:22.568952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51cc5a-cae6-41e7-8606-51968c0df72c None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.569385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51cc5a-cae6-41e7-8606-51968c0df72c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.569575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51cc5a-cae6-41e7-8606-51968c0df72c None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.571291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 464/926] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:22.576645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.576934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.577208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.594742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:22.595956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:22.666562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 463/927] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:22.689347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51cc5a-cae6-41e7-8606-51968c0df72c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:22.792421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51cc5a-cae6-41e7-8606-51968c0df72c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:22.828601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:22.829342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.829624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.829896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.830556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:22.832203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-617c8763-f031-4ac6-97f1-c1044dea4420 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:22.845310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:22.846301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:22.847185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 465/928] 10.222.0.86 () {56 vars in 1132 bytes} [Wed Jul 20 21:40:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:22.853289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:22.895725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa05f4-38cd-48a9-9377-579e837f3e26 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.896168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa05f4-38cd-48a9-9377-579e837f3e26 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.896541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa05f4-38cd-48a9-9377-579e837f3e26 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.898920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 466/929] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:22.903930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.904316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:22.904696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:22.921914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:22.922556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:22.923925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:22.932483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:22.958086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39bf4c-96ef-4fe2-b0de-8991f1ec874e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:22.998326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:22.999453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:22.999945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.000428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.001212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 467/930] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:23.001594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:23.002115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.003728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.009831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 464/931] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:23.017570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.024789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.127524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58a513b-bca6-46d6-8dcc-e615867d2c0d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.139779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde8be8-e26f-4504-863a-eacb526245ca None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.166887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.167680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.167965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.168238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.169061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:23.169372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.170735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.171195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:23.178504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.179212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 468/932] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:23.180007 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.180359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.180700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.185190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.191716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:23.194873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:23.202637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:23.203653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:23.204650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 465/933] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:23.210184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.306472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26622484-d634-4afc-ad88-9b7f10400cba None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.317779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa7798-1e5c-476e-9b01-bf5650764e1f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.344873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbbb64d-e5db-49be-9e46-5a3d067e3502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.345609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbb64d-e5db-49be-9e46-5a3d067e3502 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.345990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbb64d-e5db-49be-9e46-5a3d067e3502 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.346388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbbb64d-e5db-49be-9e46-5a3d067e3502 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.347384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbb64d-e5db-49be-9e46-5a3d067e3502 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.348986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbbb64d-e5db-49be-9e46-5a3d067e3502 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.353996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.354834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.355285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.355742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.356660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:23.357160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.358685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.369521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 466/934] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:23.375114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 469/935] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:23.376406 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.441198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:23.498899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195452a5-e461-44f9-bfb0-0c9c52fa14fa None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.547608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.548450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.548685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.548953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.558336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51174-b8fc-4e53-882c-47b4e9bdf69a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51174-b8fc-4e53-882c-47b4e9bdf69a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.558917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f51174-b8fc-4e53-882c-47b4e9bdf69a None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.560482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 470/936] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:23.561893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.563391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.565761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.566603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.567089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:23.582209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:23.583403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:23.584622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:23.584622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:23.584622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:23.584622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:23.584622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 467/937] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:23.589357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.593572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:23.594868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:23.664003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:23.676984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 471/938] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:23.700737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.724538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:23.728759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b68d4e-c8b3-4b89-92fb-84a4b4e335db None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.776789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.777901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.778622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.779137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.780750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:23.781249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.783768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.805075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 468/939] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:23.806184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.835828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183da009-7445-4f8d-9fdb-47e01aa34545 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.881337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.882067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.882366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.882716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.883765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:23.885869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:23.887647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING keystone.server.flask.common [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 21:40:23.912584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.sql.core [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 21:40:23.912584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 21:40:23.912584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [parameters: {'id': '36338a017ac34131aacf7819d4c2c244', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 21:40:23.912584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86636) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 36338a017ac34131aacf7819d4c2c244.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 36338a017ac34131aacf7819d4c2c244. Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application conn.query(q) Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 21:40:23.921618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result.read() Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 21:40:23.922476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application next(self.gen) Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application yield resource Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application next(self.gen) Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise self.value Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application session.commit() Jul 20 21:40:23.923497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.session.flush() Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application compat.raise_( Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 21:40:23.924419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 21:40:23.925359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application conn.query(q) Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result.read() Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 21:40:23.926186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [parameters: {'id': '36338a017ac34131aacf7819d4c2c244', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:23.928359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:23.930488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 36338a017ac34131aacf7819d4c2c244. Jul 20 21:40:23.932687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:23.940321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 472/940] 10.222.0.86 () {60 vars in 1140 bytes} [Wed Jul 20 21:40:23 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:23.940709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a386d-87e2-45ee-9d22-27ac62851c7e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:23.941981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:23.987708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:23.988551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:23.988832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:23.989107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:23.998982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.000466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.000990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:24.018087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:24.019146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:24.020117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:24.027440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 469/941] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:24.029837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.053722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.073532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba80a4bb-2411-4a7b-b006-48a21b572858 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.120439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.121249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.121530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.121817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.131136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.133345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.148010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656a33a-275f-478e-9380-415fd7309323 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:24.152471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:24.153407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:24.154318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 473/942] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:24.161157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.191581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.192200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.192480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.192753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.193603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:24.193911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.195748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.219157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 470/943] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:24.220222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.295839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae020665-8128-4a66-8f1a-02b1397bbf1f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.306911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.326812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.343961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.345512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.345985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.346664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.347612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:24.348313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.350705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d849e3b-4951-474b-893e-7096fa78428e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.352139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1dc02d-603a-42ca-8a99-e7d87d270487 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.395089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 474/944] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:24.400564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.401396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.401678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.401954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.404601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5456-acd2-4099-804c-2b82ec4a9e81 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.404894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5456-acd2-4099-804c-2b82ec4a9e81 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.405173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b5456-acd2-4099-804c-2b82ec4a9e81 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.406925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 475/945] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:24.412835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.413190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.413462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.416048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.417797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.432583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:24.434170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:24.439210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:24.440137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:24.441052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:24.441052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:24.441052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:24.441052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:24.443517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 471/946] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:24.451905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.527500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 476/947] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:24.543692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.587592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905abce-a8e9-47ca-a9c9-5caa4ac2e3f2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.634655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.649846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.650534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.650813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.651128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.651869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:24.652173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.653561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.664987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.670134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e229f-18c4-4658-8a35-efb1a728fa78 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.671171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 472/948] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:24.672831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.707990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.709085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.709549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.710030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.710882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.712468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:24.725014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:24.726737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:24.727650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:24.730582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 477/949] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:24.736891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.784809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20959b35-6150-480b-a2a8-cbadcac69335 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.822840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.823934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.824220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.824496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.834297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.836682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.844681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beed65a-326c-4209-b0fe-2202ed6d7295 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:24.860261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:24.861457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:24.862371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:24.865016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 473/950] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:24.872494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.888653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:24.889324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:24.889602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:24.889883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:24.890770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:24.891106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:24.893122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:24.903720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 478/951] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:24.904796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:24.943519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:24.955443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:25.003252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee1e6f-d0c1-4a83-a7dc-a5727968c158 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:25.036122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfca410-7095-4b7a-af07-1664a2880294 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:25.047757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:25.048752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.049240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.049708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.050992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:25.051704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:25.053621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:25.068993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 474/952] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:25.075181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:25.080918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:25.081607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.081890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.082165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.082895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:25.084038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:25.084685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:25.098786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 479/953] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:25.099817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:25.210944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1004e7f-44aa-4300-beb4-ca3dd3c1f296 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:25.224915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:25.237458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:25.254132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6e434-1f0c-44db-ae08-8c159aa5b9c3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:25.259733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eddfd9f-7e27-4f3c-9dcf-690bb84ceffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:25.260771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfd9f-7e27-4f3c-9dcf-690bb84ceffb None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.261147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfd9f-7e27-4f3c-9dcf-690bb84ceffb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.261525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfd9f-7e27-4f3c-9dcf-690bb84ceffb None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/d3d4323286624d928d94a6486d299eea/roles/3704a72fb24e4d2fb163fb0213f53e89` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.263913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddfd9f-7e27-4f3c-9dcf-690bb84ceffb None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=d3d4323286624d928d94a6486d299eea, role_id=3704a72fb24e4d2fb163fb0213f53e89)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:25.265976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eddfd9f-7e27-4f3c-9dcf-690bb84ceffb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:25.289926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 475/954] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:25 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/d3d4323286624d928d94a6486d299eea/roles/3704a72fb24e4d2fb163fb0213f53e89 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:40:25.305153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfce84-f688-4087-9d80-72e6d5431a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:25.306155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfce84-f688-4087-9d80-72e6d5431a50 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.306602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfce84-f688-4087-9d80-72e6d5431a50 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.307035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfce84-f688-4087-9d80-72e6d5431a50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.307986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfce84-f688-4087-9d80-72e6d5431a50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:25.309570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfce84-f688-4087-9d80-72e6d5431a50 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:25.330745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 480/955] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:25.536317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089d176-5cbc-48bc-b3a1-4676588ce504 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.536528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089d176-5cbc-48bc-b3a1-4676588ce504 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.536806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089d176-5cbc-48bc-b3a1-4676588ce504 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.538854 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 476/956] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:25.544112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.544408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.544689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.561980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:25.563226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:25.641424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 481/957] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:25.664981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089d176-5cbc-48bc-b3a1-4676588ce504 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:25.784639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089d176-5cbc-48bc-b3a1-4676588ce504 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:25.822669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:25.823732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:25.824183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:25.824604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] PATH_INFO: `/v3/services/compute` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:25.825400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:25.826917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:25.840145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:25.841514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:25.842644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:25.845227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 477/958] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:25.851307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:25.930128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:25.963299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1897a5b2-a418-48e4-a6a3-85a64d2a796d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:25.999368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:26.000276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:26.000712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:26.001154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:26.002165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:26.002683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:26.004249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03d1133-a03c-4819-bfff-2a8b8ad0c658 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:26.010964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 482/959] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:26.019334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:26.057152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:26.127644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6fadb-7f7d-483d-adcc-d42df635b46a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:26.163553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:26.164218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:26.164515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:26.164796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:26.165518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:26.165828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:26.167234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6864307a-6eb6-477c-b360-a785d97ee3f5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:26.172315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 478/960] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:26.612397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315037-11b8-4366-b4a0-d59a9a557c50 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:26.612948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315037-11b8-4366-b4a0-d59a9a557c50 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:26.612948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36315037-11b8-4366-b4a0-d59a9a557c50 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:26.614573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 483/961] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:26.620667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:26.621106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:26.621281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:26.638442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:26.639669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:26.709481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 479/962] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:26.733098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315037-11b8-4366-b4a0-d59a9a557c50 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:26.839098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36315037-11b8-4366-b4a0-d59a9a557c50 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:26.875054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:26.875686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:26.876219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:26.876498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:26.877221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:26.878647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:26.887693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:26.892209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.sql.core [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 21:40:26.892209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 21:40:26.892209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [parameters: {'user_id': 'fcda1932d67243aa9d5075d6ae8f4033', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 21:40:26.892209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86636) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application conn.query(q) Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result.read() Jul 20 21:40:26.902541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 21:40:26.903475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application next(self.gen) Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application yield resource Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application next(self.gen) Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise self.value Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application session.commit() Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 21:40:26.904423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.session.flush() Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application compat.raise_( Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 21:40:26.905335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 21:40:26.906256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application conn.query(q) Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application result.read() Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 21:40:26.907162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fcda1932d67243aa9d5075d6ae8f4033', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:26.908115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 21:40:26.908936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:26.909896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 484/963] 10.222.0.86 () {60 vars in 1131 bytes} [Wed Jul 20 21:40:26 2022] POST /identity/v3/users => generated 160 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:26.910319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:26.947969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:27.023250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a9bfd-5aad-4725-accf-b97d2a91e3a4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:27.058324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:27.059161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.059447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.059727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.067409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:27.068862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-5d479861-2397-4e25-8ee9-d5234f0dbc17 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:27.084872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:27.090090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:27.091106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 480/964] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:27.095112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:27.207276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14b60-cb37-45ed-bd64-943451ddd932 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:27.242759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:27.243472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.243757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.244044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.244847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:27.245163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:27.246850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e32cb7-18a1-44b0-a6a5-9d4bf10d251d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:27.286788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 485/965] 10.222.0.86 () {56 vars in 1101 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:27.483655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcb692-9ccd-4332-abd3-1e6221d16e8c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.484352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcb692-9ccd-4332-abd3-1e6221d16e8c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.484837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcb692-9ccd-4332-abd3-1e6221d16e8c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.486919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 481/966] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:27.493066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.493357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.493646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.515246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:27.517122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:27.521360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152f856-8768-4125-9564-42715560328e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.521650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152f856-8768-4125-9564-42715560328e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.521934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152f856-8768-4125-9564-42715560328e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.523630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 482/967] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:27.528762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.530031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.530031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.551211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:27.552409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:27.579148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:27.601262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 486/968] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:27.623001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 483/969] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:27.624626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:27.646079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:27.666290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:27.731319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b662f2-8065-47b0-a163-3b25131e3c56 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:27.754933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ea68e-767c-49b8-b85f-c07363bd8102 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:27.779944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:27.781583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.782125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.782553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.792567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:27.793381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.793662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.793936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.794879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:27.794978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:27.796282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:27.803147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:27.804130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:27.805017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 487/970] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:27.811444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:27.813040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:27.815150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:27.816964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:27.817563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 484/971] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:27.818261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:27.863855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:27.907339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:27.923415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16b403-9706-45ea-bd79-182359bb7e61 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:27.928932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d3fef-85d6-4126-90a9-41bb7e32d0dc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:27.959384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:27.960287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.960667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.960943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.961705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:27.962019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:27.964502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:27.967375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:27.968042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:27.968323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:27.968595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:27.969302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:27.969632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:27.971102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:27.978527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 488/972] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:27.978682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 485/973] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:27.981862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:27.981996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.095947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60316532-6e4f-4075-83ac-37422aa7d3b3 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.096653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a8233-765d-4d21-82ab-002692d5467c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.106930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:28.133168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.133841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.134124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.134395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.135183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:28.135464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.135952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.136829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.136940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.137171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.137515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.142643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 486/974] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:28.148731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.151801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.153255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.161103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:28.171664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:28.172601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:28.173525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:28.173525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:28.173525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:28.173525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:28.173525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 489/975] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:28.178453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.258890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c484dbe-cfca-4547-8c8f-5f331a8fb6c4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.288536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8179a-0491-413a-a5de-564ca2a3c2ab None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.293941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.294603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.294887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.295199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.296010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] RBAC: Adding query filter params (interface=admin, service_id=5458d0bb435c4d04840048d8a6b77215, region_id=RegionOne) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:28.296321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.297726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f18877-5eb4-44fd-9f7f-fe6edd18de27 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.302786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 487/976] 10.222.0.86 () {56 vars in 1249 bytes} [Wed Jul 20 21:40:28 2022] GET /identity/v3/endpoints?service_id=5458d0bb435c4d04840048d8a6b77215&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:28.308484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbc61e-6099-4760-b494-ebb98b907002 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.309180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbc61e-6099-4760-b494-ebb98b907002 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.309460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbc61e-6099-4760-b494-ebb98b907002 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.310735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 488/977] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:28.311442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbc61e-6099-4760-b494-ebb98b907002 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.322866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.323570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.323856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.324134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.324866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:28.325178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.326535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d28fb2-498a-4c72-8400-c5b59f108013 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.338965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 490/978] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:28.340199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300164c4-6670-44d9-9364-e89bd490817c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.340485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300164c4-6670-44d9-9364-e89bd490817c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.340756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300164c4-6670-44d9-9364-e89bd490817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.357050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-300164c4-6670-44d9-9364-e89bd490817c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:28.358235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-300164c4-6670-44d9-9364-e89bd490817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:28.372365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:28.393584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:28.420455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbc61e-6099-4760-b494-ebb98b907002 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.427544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 491/979] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:28.428394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300164c4-6670-44d9-9364-e89bd490817c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.458403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.459084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.459398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.460208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.460905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.462284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.468237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 489/980] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:28 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:40:28.469009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.537172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300164c4-6670-44d9-9364-e89bd490817c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.576584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.577419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.577703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.577988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.586898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.588410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.588525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7031700-b489-4f0e-95eb-d724562cad11 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:28.604780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:28.605988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:28.607592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 492/981] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:40:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:28.612060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.626500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:28.634602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.635326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.635611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.636569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.636679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.638017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.649202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:28.659112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 490/982] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:28 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:28.732183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad6a8-94b1-4397-89d4-83eb3618419d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.778516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.779238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.779526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.779801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:28.780699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:28.781019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:28.782464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:28.806708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 493/983] 10.222.0.86 () {56 vars in 1149 bytes} [Wed Jul 20 21:40:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:28.808031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:28.915605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf336251-d895-4690-ac3b-abcec87de611 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:28.951210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:28.951868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:28.952157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:28.954065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:29.017692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:29.022370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:29.023946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:29.131303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 491/984] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:28 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:29.136330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:29.203158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:29.243552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd1673-2884-4a61-9faf-43c83b389dd4 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:29.277401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:29.278175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:29.278451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:29.278726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:29.289308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:29.290744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:29.308777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:29.309888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:29.310989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:29.310989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:29.310989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:29.310989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:29.310989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 494/985] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:29.315654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:29.414976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:29.419357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd65c13-df12-4768-9595-cf81fd848233 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:29.454201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:29.454791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:29.455083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:29.455387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:29.456116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:29.456426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:29.457790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:29.469272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 492/986] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:29.474561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:29.575919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254b81f-7840-418b-85e6-55bcf089acc0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:29.592967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:29.612232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:29.613037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:29.613320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:29.613595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:29.621313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:29.622835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:29.637942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:29.639608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:29.640817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:29.643409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 495/987] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:29.644996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:29.749099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf096e-aec4-40d9-92d7-05f58c3c368c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:29.783426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:29.784104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:29.784389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:29.784667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:29.785475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:29.785791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:29.787391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a55e1-0242-4c24-b165-50efb911411f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:29.806892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 493/988] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:29.808259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:29.911717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31762-61d2-4342-8b3e-a086837a5b15 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:29.945529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:29.946188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:29.946472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:29.946751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:29.947685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] RBAC: Adding query filter params (role.id=3704a72fb24e4d2fb163fb0213f53e89, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=d3d4323286624d928d94a6486d299eea) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:29.948002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:29.949434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aeeec9-2cd8-49f1-9d92-ff38ce1e13f7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:29.963172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 496/989] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:29 2022] GET /identity/v3/role_assignments?user.id=d3d4323286624d928d94a6486d299eea&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=3704a72fb24e4d2fb163fb0213f53e89 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:30.600195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6749-7b83-418b-bf2a-116a7ab1d256 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:30.601390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6749-7b83-418b-bf2a-116a7ab1d256 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:30.601678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6749-7b83-418b-bf2a-116a7ab1d256 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:30.603251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 494/990] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:30.608574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:30.608872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:30.609147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:30.627598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:30.628811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:30.664277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:30.701306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 497/991] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:30.723947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:30.724117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016d652-60df-45ee-af19-4daafb48c492 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:30.724560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016d652-60df-45ee-af19-4daafb48c492 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:30.724560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016d652-60df-45ee-af19-4daafb48c492 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:30.726610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 495/992] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:30.731689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:30.731975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:30.732243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:30.749828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:30.751008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:30.764822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:30.824536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 496/993] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:30.833814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481380-0a77-4dc4-85d3-e62b9787846d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:30.846082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:30.872923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:30.873653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:30.873935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:30.874211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:30.874871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:30.876261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:30.888674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:30.889686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:30.890532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 498/994] 10.222.0.86 () {56 vars in 1126 bytes} [Wed Jul 20 21:40:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:30.895827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:30.934850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:30.956442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c434b-7390-4589-aad0-16f8721b1fe8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:30.991713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:30.992501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:30.992782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:30.993049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:30.993694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:30.995069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:31.006710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:31.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:31.010147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:31.012776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 497/995] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:31.013605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:31.017935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d36af5-029d-4c00-b7ce-1034732b3d48 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.028320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:31.056863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.057544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.057828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.058110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.058828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:31.059169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.060545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.066351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 499/996] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:31.072154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:31.134931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635d011-2971-4a3f-9399-5aec63886d36 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.173331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.174015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.174300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.174575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.175313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] RBAC: Adding query filter params (name=compute) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:31.175621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.177005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.178451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:31.182484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46673fa8-911f-486f-b67d-9cd7573f8449 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.182961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 498/997] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:31.188370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:31.227188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.228379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.228645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.228923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.229639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:31.229952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.231342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.236742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 500/998] 10.222.0.86 () {56 vars in 1122 bytes} [Wed Jul 20 21:40:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:31.242234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:31.271054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:31.296896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5345cf-fb19-460a-af76-eee6f8a8ce5f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.335422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.336084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.336371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.336646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.337360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:31.337671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.339044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.343998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 499/999] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:31.349667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:31.350906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c2b8e-31f9-48cc-8c2d-ccaf2d33137f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.387202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.388138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.388596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.389042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.390069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.391516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.419096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 501/1000] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:40:31.469504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee73fbf-d17d-454b-a9a5-9e964bf2cff9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.514705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.515410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.515697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.515973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.516776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] RBAC: Adding query filter params (interface=public, service_id=30277863e6034c7eb9b4cea9f5d9f8f8, region_id=RegionOne) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:31.517385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.518798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3077b2bb-edb8-4273-a724-aa806119123d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.521659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:31.531267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 500/1001] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:31 2022] GET /identity/v3/endpoints?service_id=30277863e6034c7eb9b4cea9f5d9f8f8&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:31.532637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:31.559798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:31.645008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbe0e6-543a-4b89-a7ba-443ea427ffe0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:31.680367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7d234-62ab-4503-a304-ca57cd0389e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:31.681049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7d234-62ab-4503-a304-ca57cd0389e5 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:31.681306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7d234-62ab-4503-a304-ca57cd0389e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:31.681581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7d234-62ab-4503-a304-ca57cd0389e5 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:31.682268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7d234-62ab-4503-a304-ca57cd0389e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:31.683671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7d234-62ab-4503-a304-ca57cd0389e5 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:31.690399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 502/1002] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:31 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:40:32.045060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98157-f2e4-45ab-825d-b92f48ce64a7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:32.045588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98157-f2e4-45ab-825d-b92f48ce64a7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:32.045588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d98157-f2e4-45ab-825d-b92f48ce64a7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:32.047192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 501/1003] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:32.052311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:32.052605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:32.052886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:32.071510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:32.072708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:32.147241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 503/1004] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:32.169117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98157-f2e4-45ab-825d-b92f48ce64a7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:32.274408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d98157-f2e4-45ab-825d-b92f48ce64a7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:32.308871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:32.309592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:32.309877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:32.310156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] PATH_INFO: `/v3/services/image` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:32.310809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:32.312227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:32.324207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:32.325241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:32.326135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:32.331119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 502/1005] 10.222.0.86 () {56 vars in 1105 bytes} [Wed Jul 20 21:40:32 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:32.331581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:32.349834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:32.435870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69924b9d-d7f2-410a-bb08-cee478d3b2e9 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:32.471561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:32.472227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:32.472508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:32.472784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:32.473491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] RBAC: Adding query filter params (name=image) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:32.473798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:32.475174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd919549-55fe-4dd2-b8fa-029f8e42de04 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:32.485589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 504/1006] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:32.487141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:32.556114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:32.595922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7fd469-778d-4ca0-8a30-1141204296da None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:32.629235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:32.629893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:32.630170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:32.630440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:32.631168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] RBAC: Adding query filter params (type=image) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:32.631649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:32.632891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d1277-020a-4073-a335-7852bc46a5f5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:32.637452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 503/1007] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:33.492252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd11b35-70ee-4733-ac61-53bae5d2c68d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:33.492770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd11b35-70ee-4733-ac61-53bae5d2c68d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:33.492851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd11b35-70ee-4733-ac61-53bae5d2c68d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:33.495100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 505/1008] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:33.499416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:33.499838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:33.500025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:33.517569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:33.518767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:33.585755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 504/1009] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:33.608591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd11b35-70ee-4733-ac61-53bae5d2c68d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:33.692815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:33.715282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd11b35-70ee-4733-ac61-53bae5d2c68d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:33.753732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db50ef-aebb-4232-aa58-a746b5ae2f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:33.754783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db50ef-aebb-4232-aa58-a746b5ae2f7a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:33.755242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db50ef-aebb-4232-aa58-a746b5ae2f7a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:33.755667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db50ef-aebb-4232-aa58-a746b5ae2f7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:33.757122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db50ef-aebb-4232-aa58-a746b5ae2f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:33.758726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db50ef-aebb-4232-aa58-a746b5ae2f7a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:33.760864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 506/1010] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:33.766553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:33.776178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc85bd-177b-4a9a-a04e-711fbb7b2916 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:33.776498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc85bd-177b-4a9a-a04e-711fbb7b2916 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:33.776727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc85bd-177b-4a9a-a04e-711fbb7b2916 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:33.778227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 507/1011] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:33.783114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:33.783402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:33.783679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:33.800468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:33.801653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:33.828217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:33.872084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 508/1012] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:33.876173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed33b7b-ee9f-42f0-9002-ff0351dcec5d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:33.894526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:33.911434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:33.912092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:33.912378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:33.913216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:33.913904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:33.915363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:33.942818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 505/1013] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:33 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:40:33.992732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:34.018964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457cfa67-b2a7-4841-b12d-d247fc007a49 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:34.057037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:34.057851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.058131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.058401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.059096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:34.060526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:34.073435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:34.074546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:34.080981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 509/1014] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:34.084020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:34.192474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef07a-fbf3-4dc2-bf1e-435892b5d073 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:34.203387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:34.227959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:34.228626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.228901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.229174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.229887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] RBAC: Adding query filter params (name=compute) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:34.230188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:34.231602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:34.236816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 506/1015] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:34.243250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:34.346763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08558d34-9dd4-4949-9156-1f9d43addc11 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:34.381370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:34.382651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.382875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.383182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.383897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] RBAC: Adding query filter params (type=compute) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:34.384210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:34.385572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd2f4d8-3765-467b-8690-ffce83b3665f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:34.390665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 510/1016] 10.222.0.86 () {56 vars in 1112 bytes} [Wed Jul 20 21:40:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:34.396153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:34.500932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5ee7b-3ca9-42c4-9e7b-da6c24211bde None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:34.536204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187b09e-9a60-4604-ade2-7bdd750b293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:34.537284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b09e-9a60-4604-ade2-7bdd750b293c None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.538557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b09e-9a60-4604-ade2-7bdd750b293c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.539093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187b09e-9a60-4604-ade2-7bdd750b293c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.540071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b09e-9a60-4604-ade2-7bdd750b293c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:34.541667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187b09e-9a60-4604-ade2-7bdd750b293c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:34.561972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 507/1017] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:34.696285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59834-6ecb-47a5-baac-c4a5aaa19353 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.696809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59834-6ecb-47a5-baac-c4a5aaa19353 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.696809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59834-6ecb-47a5-baac-c4a5aaa19353 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.698300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 511/1018] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:34.703909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721c798-7a72-400e-9b5c-8806939fc35f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.704161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721c798-7a72-400e-9b5c-8806939fc35f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.704434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721c798-7a72-400e-9b5c-8806939fc35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.721755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c721c798-7a72-400e-9b5c-8806939fc35f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:34.722963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c721c798-7a72-400e-9b5c-8806939fc35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:34.749841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:34.805287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 508/1019] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:34 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:34.827967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b59834-6ecb-47a5-baac-c4a5aaa19353 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:34.839655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:34.936985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b59834-6ecb-47a5-baac-c4a5aaa19353 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:34.972639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45444e-ef7c-4bc4-828e-5b778a165992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:34.973312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45444e-ef7c-4bc4-828e-5b778a165992 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:34.973590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45444e-ef7c-4bc4-828e-5b778a165992 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:34.973860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45444e-ef7c-4bc4-828e-5b778a165992 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:34.974582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45444e-ef7c-4bc4-828e-5b778a165992 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:34.976659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45444e-ef7c-4bc4-828e-5b778a165992 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:34.992199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 512/1020] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:34 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:36.057827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2b5d5-501b-46fa-802a-ea225cc72477 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:36.058386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2b5d5-501b-46fa-802a-ea225cc72477 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:36.058386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2b5d5-501b-46fa-802a-ea225cc72477 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:36.059919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 509/1021] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:36.064902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:36.065288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:36.065485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:36.086759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:36.086759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:36.152572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:36.168358 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 513/1022] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:36.192732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2b5d5-501b-46fa-802a-ea225cc72477 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:36.296146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2b5d5-501b-46fa-802a-ea225cc72477 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:36.330823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a1935a-f2de-4d12-a849-2c5fd6008bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:36.331580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1935a-f2de-4d12-a849-2c5fd6008bcc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:36.331859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1935a-f2de-4d12-a849-2c5fd6008bcc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:36.332130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1935a-f2de-4d12-a849-2c5fd6008bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:36.333305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1935a-f2de-4d12-a849-2c5fd6008bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:36.335387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a1935a-f2de-4d12-a849-2c5fd6008bcc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:36.337144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 510/1023] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:36.342667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:36.444276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8104e-f3df-46b6-9ccd-64f8d4d486da None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:36.479929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267605f-a33f-4eef-9e12-fdf23292fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:36.480610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267605f-a33f-4eef-9e12-fdf23292fb05 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:36.480898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267605f-a33f-4eef-9e12-fdf23292fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:36.481174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267605f-a33f-4eef-9e12-fdf23292fb05 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:36.481904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267605f-a33f-4eef-9e12-fdf23292fb05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:36.483373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267605f-a33f-4eef-9e12-fdf23292fb05 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:36.505285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 514/1024] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:40:36 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:36.827362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039d81d-63f6-4ec7-8be8-d69ab9bfee5d None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:36.827740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039d81d-63f6-4ec7-8be8-d69ab9bfee5d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:36.827951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039d81d-63f6-4ec7-8be8-d69ab9bfee5d None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:36.829394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 511/1025] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:36.834537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7f6e-1120-4aa5-a252-0243bb4b3d55 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:36.834980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7f6e-1120-4aa5-a252-0243bb4b3d55 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:36.835293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d7f6e-1120-4aa5-a252-0243bb4b3d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:36.852837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-838d7f6e-1120-4aa5-a252-0243bb4b3d55 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:36.854017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-838d7f6e-1120-4aa5-a252-0243bb4b3d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:36.900297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:36.927654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 515/1026] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:36.953795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039d81d-63f6-4ec7-8be8-d69ab9bfee5d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:36.983047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.062323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a615e7-852a-4872-9177-bd0fddec192c None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.062688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a615e7-852a-4872-9177-bd0fddec192c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.062873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a615e7-852a-4872-9177-bd0fddec192c None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.064426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 516/1027] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:37.069351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.069922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.069922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.074547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039d81d-63f6-4ec7-8be8-d69ab9bfee5d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.093589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:37.094782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:37.116417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.117801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.118235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.118633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.133197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:37.143500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:37.144496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:37.145428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:37.152343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 512/1028] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Jul 20 21:40:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:37.152705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.167108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 517/1029] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:37.189282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.246427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.263437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c6c36-82a1-4fc4-9051-edd3a2519a7d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.299845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.300520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.300798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.301069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.301260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270940e1-2291-43ad-9775-6fc6c1dc0002 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.301815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:37.302119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.303705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.316470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 513/1030] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:37.316674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.317520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.339396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.340139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.340420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.340690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] PATH_INFO: `/v3/services/image` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.341341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.342697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:37.354896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:37.355942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:37.356820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 518/1031] 10.222.0.86 () {56 vars in 1105 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:37.362269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.424061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144db119-9f90-4e36-bddd-390ea338cb7f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.457388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.458171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.458450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.458726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.466148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3dfac-328b-4321-9670-3e682e1ad014 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.469665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.471132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.477990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:37.489844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:37.490829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:37.491830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:37.491830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:37.491830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:37.491830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:37.491830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 514/1032] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:37.496866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.508373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.509062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.509338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.509932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.510944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] RBAC: Adding query filter params (name=image) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:37.511280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.513165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.519480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 519/1033] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:37.525338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.561677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.609242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca760f-44ac-48e8-aaf1-126b9eaaab87 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.631279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1774e21-852e-44f4-a7a1-be8cf514eea2 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.646840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.647795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.648081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.648362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.649095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:37.649404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.650776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.663440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 515/1034] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:37.668844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.673940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.674627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.674913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.675245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.676044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] RBAC: Adding query filter params (type=image) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:37.676359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.677816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.684703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 520/1035] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:37.690631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.734297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.778135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d649d8-6647-4393-b128-dd3974ec487f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.799526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.802485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62a1d0-263d-4cbd-ae65-72f8111e3ed3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.818826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.819820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.820100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.820371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.827567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.829006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.839696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.840350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.840633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.840907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:37.841696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] RBAC: Adding query filter params (interface=public, service_id=f0da9a7ecef24f99b682062b90573bb4, region_id=RegionOne) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:37.842009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:37.843407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:37.843823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:37.844811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:37.845805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:37.849674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 516/1036] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:37.851335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.851442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 521/1037] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/endpoints?service_id=f0da9a7ecef24f99b682062b90573bb4&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:37.856488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:37.953435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a7e58-5720-4e95-9226-43aec6673b01 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.964931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8211a-8d88-4038-a465-e315a701336f None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:37.977153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:37.998166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:37.998838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:37.999147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:37.999429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.000275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:38.000587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:38.002014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:38.005533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:38.006204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.006519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.006796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.007525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:38.008878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:38.016154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 522/1038] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:40:38.031766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 517/1039] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:40:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:38.039304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.072340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:38.160958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f599-ac4c-4ec4-9944-fb0886063881 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:38.199238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:38.199896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.200180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.200458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.270714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:38.272758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:38.300165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:38.375630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 523/1040] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 21:40:38.381150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.482623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139db00-7967-4c99-badc-52300d30ec41 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:38.518721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:38.519564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.519849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.520124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.531731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:38.533118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-06ff4109-5fd7-43d9-8fec-99b4438ff003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:38.550863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:38.551869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:38.553473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:38.553473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:38.553473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:38.553473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:38.553473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 518/1041] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:38.558079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.629301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9571d-b318-4c2d-a286-0e91562bd271 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.629588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9571d-b318-4c2d-a286-0e91562bd271 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.629864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9571d-b318-4c2d-a286-0e91562bd271 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.632101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 519/1042] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:38.638313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.638604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.638884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.657326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:38.658528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:38.662733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75b3f4-80bc-408f-9ddd-fa69277c6ab0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:38.683804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:38.695320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:38.704262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:38.704931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.705215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.705493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.706286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:38.706524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:38.707933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:38.724749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 524/1043] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:38.725700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.732443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 520/1044] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:38.756311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.833873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2fc83-faf2-4edd-a5f2-3c7e0a8953bf None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:38.865382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fabe43-f079-4226-a91b-9c01e73de976 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:38.870806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:38.871631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.871915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.872189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.880204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:38.882336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:38.896975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:38.898241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:38.899202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 525/1045] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:38.903878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.908185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:38.909289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:38.909718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:38.910138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:38.921551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:38.930043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:38.930989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:38.931910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 521/1046] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:38.936989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:38.956428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:38.961689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:39.018700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef82ca-e60b-4988-b062-c0abb0c23b57 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.044981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a28f53-c360-484f-8291-98c121fb375c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.056437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.057378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.057811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.058260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.059231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:39.059691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.061277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.074331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 526/1047] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:39.080527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:39.085351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.086146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.086422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.086693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.087574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:39.087880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.089262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.098105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 522/1048] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:39.103764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:39.191785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:39.203429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dfb3a-4b68-4321-92ca-33ad55ee0464 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.213086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd104c3-9e56-43cc-aed9-db20aad82912 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.216941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:39.247376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.248049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.248311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.248579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.249401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] RBAC: Adding query filter params (role.id=3704a72fb24e4d2fb163fb0213f53e89, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=e06439f92e494fb4a68ffee7bdb0ffc5) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:39.249710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.251253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.252328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.253688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.254671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.255477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] PATH_INFO: `/v3/users/de9640f94250421b91f89b262ef88e5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.257773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 527/1049] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:39 2022] GET /identity/v3/role_assignments?user.id=e06439f92e494fb4a68ffee7bdb0ffc5&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=3704a72fb24e4d2fb163fb0213f53e89 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:39.281924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] RBAC: Authorizing `identity:get_user(user_id=de9640f94250421b91f89b262ef88e5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.283456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.285467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 523/1050] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:39 2022] GET /identity/v3/users/de9640f94250421b91f89b262ef88e5c => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:39.291317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:39.400868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a7dd5-a3f4-44c5-ab0d-19e260cbb719 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.437828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.438476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.438757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.439029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.481546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:39.511388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.512844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.618037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 528/1051] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:39 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:39.619536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:39.720669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45050094-6d3c-421c-8323-8bd627e88efb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.755835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ed583-30de-4950-ab64-e3b2335b2637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.757249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed583-30de-4950-ab64-e3b2335b2637 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.757688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed583-30de-4950-ab64-e3b2335b2637 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.757981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed583-30de-4950-ab64-e3b2335b2637 None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.759222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ed583-30de-4950-ab64-e3b2335b2637 None admin] RBAC: Authorizing `identity:get_project(project_id=fee14ca70d154433bcb19b8a55bf516e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.760668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ed583-30de-4950-ab64-e3b2335b2637 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.767350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 524/1052] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:40:39 2022] GET /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:39.768604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:39.839658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:39.871270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d148d62-6e2d-4bfd-af51-57536cf9d572 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:39.906230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:39.906888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:39.907198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:39.907478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:39.908306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=fee14ca70d154433bcb19b8a55bf516e, user.id=de9640f94250421b91f89b262ef88e5c) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:39.908717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:39.910042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fad33-b9a7-45f7-b10c-e769d0ba95f8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:39.923102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 529/1053] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:39 2022] GET /identity/v3/role_assignments?user.id=de9640f94250421b91f89b262ef88e5c&scope.project.id=fee14ca70d154433bcb19b8a55bf516e&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:40.086906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49d34e-10a2-41bc-84fd-6f6e2b495b30 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:40.087187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49d34e-10a2-41bc-84fd-6f6e2b495b30 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:40.087469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49d34e-10a2-41bc-84fd-6f6e2b495b30 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:40.089942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 525/1054] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:40.094356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:40.094780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:40.094880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:40.111709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:40.112882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:40.180093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 530/1055] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:40.203341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49d34e-10a2-41bc-84fd-6f6e2b495b30 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:40.212105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:40.303151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49d34e-10a2-41bc-84fd-6f6e2b495b30 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:40.337206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:40.337910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:40.338190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:40.338461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] PATH_INFO: `/v3/services/image` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:40.339138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:40.340504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:40.352542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:40.353561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 21:40:40.354411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:40.359001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 526/1056] 10.222.0.86 () {56 vars in 1105 bytes} [Wed Jul 20 21:40:40 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:40.360059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:40.429756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:40.466145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97ad93-ff2c-45e8-a82a-d8c492d66bbc None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:40.500422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:40.501085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:40.501374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:40.501645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:40.502355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] RBAC: Adding query filter params (name=image) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:40.502662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:40.504062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:40.514169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 531/1057] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:40.515768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:40.586577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:40.619470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f941f-78ff-4d1e-8dee-f8cbb05cedf5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:40.653447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:40.654214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:40.654386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:40.654661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:40.655402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] RBAC: Adding query filter params (type=image) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:40.655713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:40.657110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003abcd-19d3-4aa8-b9be-b62581b48986 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:40.666569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 527/1058] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:40.668152 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:40.774556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00d992-24a5-4565-98b8-dabd35777185 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:40.803473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:40.810827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4a19b-b50a-4014-83b0-c4fb3a4bd73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:40.811769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4a19b-b50a-4014-83b0-c4fb3a4bd73f None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:40.812259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4a19b-b50a-4014-83b0-c4fb3a4bd73f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:40.812768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4a19b-b50a-4014-83b0-c4fb3a4bd73f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:40.813750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4a19b-b50a-4014-83b0-c4fb3a4bd73f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:40.815354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4a19b-b50a-4014-83b0-c4fb3a4bd73f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:40.841941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 532/1059] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:40:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:40:41.323504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af293f-f678-49ee-b7e1-e54454ff29c6 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.323701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af293f-f678-49ee-b7e1-e54454ff29c6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.323955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af293f-f678-49ee-b7e1-e54454ff29c6 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.325322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 528/1060] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:41.330576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.331035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.331143 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.348531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:41.349703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:41.425881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 533/1061] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:41.448956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af293f-f678-49ee-b7e1-e54454ff29c6 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:41.552109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af293f-f678-49ee-b7e1-e54454ff29c6 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:41.587321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:41.588057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.588334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.588601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.590476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:41.601238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:41.609741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:41.610664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:41.612123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:41.615392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 529/1062] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Jul 20 21:40:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:41.618103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:41.718945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9238ad9-cd28-4bce-9bd4-c5e478686877 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:41.732290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:41.756132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:41.757365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.757648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.757922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.758668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:41.758975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:41.760369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8a6f8-baf9-4dfb-abab-6a2f642f5834 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:41.773276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 534/1063] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:40:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:41.774488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:41.881060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeae6b1-a96c-4c5f-bfd6-18e0b7df9dba None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:41.920666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:41.921745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.922223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.922742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.934105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:41.935891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-056ee0ed-675d-497a-9f95-fd891dcc8298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:41.959615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:41.960815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:41.969946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:41.969946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:41.969946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:41.969946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:41.969946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 530/1064] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:41.970264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f44f8c-5659-4251-831d-7af5bec0246d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.970264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f44f8c-5659-4251-831d-7af5bec0246d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.970264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f44f8c-5659-4251-831d-7af5bec0246d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.970264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 535/1065] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:41.972652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f44f8c-5659-4251-831d-7af5bec0246d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:41.972966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:41.973254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:41.973529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:41.991843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:41.993060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:42.045581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.083749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 531/1066] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:42.103390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.106149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f44f8c-5659-4251-831d-7af5bec0246d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.149204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.149856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.150142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.150412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.151167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:42.151475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.152834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.173216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 536/1067] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:42.174640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.214357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.220635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247480a4-5b08-4767-8aed-02f23e32e21a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.259190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.260087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.260380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.260657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.271795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:42.279590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:42.280720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:42.283982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 532/1068] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:42.289196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.301562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539b881-a99e-4dc1-b3eb-41b2c0ff9628 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.326295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.342599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.343437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.343717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.343989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.353950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.355884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:42.375035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:42.376153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 537/1069] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:42.383119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.406276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f064b-c190-4aa5-96bd-5b8f90738b62 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.452570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.453219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.453504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.453772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.454520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:42.454822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.456236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.468163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 533/1070] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:42.475733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.497770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.510135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099ff67-96df-4e82-9819-a88872cc6de7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.548487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.549172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.549455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.549733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.550586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:42.550899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.552362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.571912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 538/1071] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:42.578750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.599922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.606482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02caeb9e-18e5-413d-9f21-639b225c27fb None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.656644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.657590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.657868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.658136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] PATH_INFO: `/v3/users/de9640f94250421b91f89b262ef88e5c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.659397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=de9640f94250421b91f89b262ef88e5c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.660831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.662470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 534/1072] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/users/de9640f94250421b91f89b262ef88e5c => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:42.669625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.698463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ce515-825d-4475-ab04-de6fe8d05982 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.744671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.745466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.745743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.746022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.757398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.758850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:42.777597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:42.778551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:42.779571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:42.779571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:42.779571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:42.779571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:42.779571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 539/1073] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:42.784572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.788759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.793376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb43ec-3c11-47b7-a5e2-5712268b8bc5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.830477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.831502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.831805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.832061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.833370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] RBAC: Authorizing `identity:get_project(project_id=fee14ca70d154433bcb19b8a55bf516e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.834888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.842001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 535/1074] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:42.843173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.879985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:42.892933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2b99f-c21c-425d-9f7c-76d5ee8e99db None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.931008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.931614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.931888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.932160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:42.932887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:42.933197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:42.934560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:42.951305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 540/1075] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:42.952377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:42.958888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764fc39-b349-4c45-8a1c-1c4e56680759 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:42.997299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627fdb7-9e8d-4f53-b798-a04c64bc051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:42.998516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627fdb7-9e8d-4f53-b798-a04c64bc051e None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:42.998956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627fdb7-9e8d-4f53-b798-a04c64bc051e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:42.999263 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627fdb7-9e8d-4f53-b798-a04c64bc051e None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e/users/de9640f94250421b91f89b262ef88e5c/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.001269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627fdb7-9e8d-4f53-b798-a04c64bc051e None admin] RBAC: Authorizing `identity:create_grant(project_id=fee14ca70d154433bcb19b8a55bf516e, user_id=de9640f94250421b91f89b262ef88e5c, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.002720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627fdb7-9e8d-4f53-b798-a04c64bc051e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.016850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 536/1076] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:42 2022] PUT /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e/users/de9640f94250421b91f89b262ef88e5c/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:40:43.018733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:43.061933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810df1d-ef0b-4a9d-a274-b484b15e7713 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.062215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810df1d-ef0b-4a9d-a274-b484b15e7713 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.062487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810df1d-ef0b-4a9d-a274-b484b15e7713 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.064014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 537/1077] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:43.064131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f92552-3627-490e-85cc-e6554979c7c6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:43.072832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.073241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.073583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.095843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:43.097087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:43.110499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:43.111607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.111891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.112165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.120580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.122046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:43.138444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:43.139509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:43.140442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 541/1078] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:43.146024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:43.179111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 538/1079] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:43.200933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:43.227875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:43.257440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159b61-664b-4974-810e-e386f2e538e0 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:43.282836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:43.295697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:43.296366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.296648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.296922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.297854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:43.298031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.299496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.310714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1bc35-3d84-45d5-84b2-84faeebc7d2d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:43.317085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 542/1080] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:43.318067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:43.354037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:43.354692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.354975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.355277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.425500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.426968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.430758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd76a4-ca63-4fca-8813-e1d6c343d39d None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:43.454540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:43.469717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:43.470609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.470893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.471198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/e06439f92e494fb4a68ffee7bdb0ffc5/roles/3704a72fb24e4d2fb163fb0213f53e89` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.473463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=e06439f92e494fb4a68ffee7bdb0ffc5, role_id=3704a72fb24e4d2fb163fb0213f53e89)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.474900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.490552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 543/1081] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:43 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/e06439f92e494fb4a68ffee7bdb0ffc5/roles/3704a72fb24e4d2fb163fb0213f53e89 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:40:43.548927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 539/1082] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 21:40:43.550685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:43.603835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:43.670601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a50f9-3388-405c-a18b-7e0c041e2af7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:43.705908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:43.706621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.706907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.707237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.717930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.719412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-7f8fc6c4-8266-4a72-a797-313fc349a696 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:43.737260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:43.738225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:43.739213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:43.739213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:43.739213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:43.739213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:43.739213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 544/1083] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:43.745440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:43.846211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2dd7d-58ca-4fe3-971c-b0c874e04727 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:43.870185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:43.882134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:43.882804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:43.883111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:43.883393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:43.884126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:43.884438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:43.885806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c6b8e-00db-445e-9515-9e248064cf9a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:43.898153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 540/1084] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:45.077606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a16d7-6bc9-4d3c-9234-f708b601adc5 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.078130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a16d7-6bc9-4d3c-9234-f708b601adc5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.078130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a16d7-6bc9-4d3c-9234-f708b601adc5 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.079551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 545/1085] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:45.085508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.085913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.086193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.102588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:45.103785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:45.170611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 541/1086] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:45.195167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a16d7-6bc9-4d3c-9234-f708b601adc5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.288643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:45.296014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a16d7-6bc9-4d3c-9234-f708b601adc5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:45.328711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:45.329414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.329693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.329964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.340849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:45.348315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:45.349335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:45.350205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 546/1087] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:45.355751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.452479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd415e29-cffa-4acb-aacc-db3974b7302d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:45.466380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:45.486896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:45.487591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.487874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.488150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.488899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:45.489209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:45.491258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f67bff-546c-4e6c-98c6-a3e4086f6037 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:45.499108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 542/1088] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:45.504766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.558059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b1e17-617a-4194-831f-2185fc920c5f None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.558346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b1e17-617a-4194-831f-2185fc920c5f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.558624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b1e17-617a-4194-831f-2185fc920c5f None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.560028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 543/1089] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:40:45.565016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.565359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.565714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.582565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:45.583721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:45.604728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b5c88-0ff2-4140-8326-f78ff69e24e7 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:45.639659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:45.640528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.640809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.641089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] PATH_INFO: `/v3/users/de9640f94250421b91f89b262ef88e5c` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.642275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] RBAC: Authorizing `identity:get_user(user_id=de9640f94250421b91f89b262ef88e5c)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:45.643739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:45.645381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 547/1090] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/users/de9640f94250421b91f89b262ef88e5c => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:45.651156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.655539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 544/1091] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:40:45.674993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:45.679696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.750123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c767e1-bbbe-41f1-bfe5-ecb2866e231c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:45.778224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0546f0-007e-4076-a513-f2d8a8ad4b83 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:45.783512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:45.784158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.784437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.784711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.785464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:45.814211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:45.815018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.815329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.815600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.825738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:40:45.834033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:45.835322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:45.836192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 545/1092] 10.222.0.86 () {56 vars in 1120 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:45.841188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.852834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:45.854292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:45.878092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:45.940862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b6d87-9810-4911-9293-ffc2dbfa1641 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:45.953725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 548/1093] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 21:40:45.959613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:45.979943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:45.980611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:45.980888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:45.981157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:45.981897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:45.982200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:45.983630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:45.992440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 546/1094] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:45.998002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:46.025143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:46.057226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2265ef6-e8ef-48f6-963b-bc1fc16b1a9c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.080248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:46.094043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.094838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.095146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.095422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.096604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] RBAC: Authorizing `identity:get_project(project_id=fee14ca70d154433bcb19b8a55bf516e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.098020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.099915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dccd7-19e0-438e-9a0b-2f06fbb079ed None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.100565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 549/1095] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:46.105665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:46.136150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.136942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.137234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.137506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.148100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.149547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:46.166508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:46.167477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:46.168403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:46.168403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:46.168403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:46.168403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:46.168403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 547/1096] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:40:46.173416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:46.205713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4acb5-c412-453e-96f3-56800b0516de None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.238723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.239405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.239690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.239965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.241285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=fee14ca70d154433bcb19b8a55bf516e, user.id=de9640f94250421b91f89b262ef88e5c) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:46.241596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.243015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.248811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 550/1097] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:46 2022] GET /identity/v3/role_assignments?user.id=de9640f94250421b91f89b262ef88e5c&scope.project.id=fee14ca70d154433bcb19b8a55bf516e&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:46.259081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:46.280616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7125d-24e6-4619-b906-dbd882e3cfdc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.317274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.317948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.318230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.318505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.320024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:46.320336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.321736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.334734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 548/1098] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:40:46.340511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:46.404696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:46.449835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b565a-8543-4096-9aca-aab6842fb0c9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.483112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.483897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.484174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.484443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.492202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.493615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:40:46.508391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:46.509605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:40:46.511203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 551/1099] 10.222.0.86 () {56 vars in 1093 bytes} [Wed Jul 20 21:40:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:46.515349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:46.619991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd9dc8-5059-4907-866f-afdde052c542 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.653909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.654575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.654906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.655162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.656005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:46.656306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.657743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.666234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:46.674820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 549/1100] 10.222.0.86 () {56 vars in 1133 bytes} [Wed Jul 20 21:40:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:46.680537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:46.780156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f1e03-2d37-45da-957e-fb0196333de9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:46.796434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:46.814115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:46.814773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:46.815081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:46.815362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:46.882478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:46.883956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:46.981488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 552/1101] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:46.987224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:47.087622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18a405-ecb0-4a22-8db2-6e56fea8ecf9 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:47.122344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:47.123155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:47.123438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:47.123713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:47.134174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:47.135665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:40:47.152599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:47.153662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:47.154594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:40:47.154594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:40:47.154594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:40:47.154594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:47.154594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 550/1102] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:47.159550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:47.176473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:47.256631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f7e70-1b45-4f0e-b327-a17bc5d98f3e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:47.289962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:47.290615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:47.290891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:47.291193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:47.291911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:47.292214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:47.293567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:47.305083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 553/1103] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:47.310621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:47.359777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:47.411639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdc55d-8396-4241-b6c5-20587696ea9d None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:47.445562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecad28-2388-425b-af79-535f3c6311da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:47.446345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecad28-2388-425b-af79-535f3c6311da None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:47.446626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecad28-2388-425b-af79-535f3c6311da None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:47.446930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecad28-2388-425b-af79-535f3c6311da None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:47.453559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecad28-2388-425b-af79-535f3c6311da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:47.455014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecad28-2388-425b-af79-535f3c6311da None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-beecad28-2388-425b-af79-535f3c6311da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:40:47.468646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:40:47.469735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:40:47.470709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:40:47.474626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 551/1104] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:40:47.476132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:47.551224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:47.574579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825721e-3f73-45c3-911e-57e2cdcf4ac2 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:47.607394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:47.608051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:47.608332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:47.608604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:47.609388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:47.609696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:47.611382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd6555-cf18-404f-8553-4c313000a119 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:47.623190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 554/1105] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:47.629174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecad28-2388-425b-af79-535f3c6311da None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:47.727793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecad28-2388-425b-af79-535f3c6311da None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:47.738141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:47.761371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:47.762024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:47.762308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:47.762586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:47.763452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] RBAC: Adding query filter params (role.id=3704a72fb24e4d2fb163fb0213f53e89, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=e06439f92e494fb4a68ffee7bdb0ffc5) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:47.763770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:47.765199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42472f2-c2f7-413d-8251-d107f1894e3e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:47.771006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 552/1106] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:47 2022] GET /identity/v3/role_assignments?user.id=e06439f92e494fb4a68ffee7bdb0ffc5&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=3704a72fb24e4d2fb163fb0213f53e89 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:48.316094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91535-740d-4d63-8875-aa8c9f43dfe2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:48.316628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91535-740d-4d63-8875-aa8c9f43dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:48.316628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91535-740d-4d63-8875-aa8c9f43dfe2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:48.318014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 555/1107] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:48.324344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:48.324786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:48.324885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:48.342322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:48.343483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:48.412034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 553/1108] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:48.437155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be91535-740d-4d63-8875-aa8c9f43dfe2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:48.540728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be91535-740d-4d63-8875-aa8c9f43dfe2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:48.575512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a2d0d-2d33-4da3-b33f-df3863d89a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:48.576305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a2d0d-2d33-4da3-b33f-df3863d89a1c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:48.576576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a2d0d-2d33-4da3-b33f-df3863d89a1c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:48.576848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a2d0d-2d33-4da3-b33f-df3863d89a1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:48.578015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a2d0d-2d33-4da3-b33f-df3863d89a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:48.579509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a2d0d-2d33-4da3-b33f-df3863d89a1c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:48.581145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 556/1109] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:48.587584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:48.687887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87365cc2-dcc8-4705-8332-25b39fe8e899 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:48.690065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:48.721683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65199e8c-0f38-4d98-8812-079a28e69758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:48.722353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65199e8c-0f38-4d98-8812-079a28e69758 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:48.722634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65199e8c-0f38-4d98-8812-079a28e69758 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:48.722910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65199e8c-0f38-4d98-8812-079a28e69758 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:48.723657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65199e8c-0f38-4d98-8812-079a28e69758 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:48.725606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65199e8c-0f38-4d98-8812-079a28e69758 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:48.745633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 554/1110] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:40:48 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:40:50.804319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15c67-ac26-46f4-b2ee-bf83ab89ecdc None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:50.804881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15c67-ac26-46f4-b2ee-bf83ab89ecdc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:50.804881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d15c67-ac26-46f4-b2ee-bf83ab89ecdc None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:50.806217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 557/1111] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:50.811479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:50.811909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:50.812033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:50.829043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:50.830223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:50.894834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 555/1112] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:50.917943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d15c67-ac26-46f4-b2ee-bf83ab89ecdc None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:50.921707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:51.019348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d15c67-ac26-46f4-b2ee-bf83ab89ecdc None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:51.053651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:51.054549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:51.054785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:51.055085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] PATH_INFO: `/v3/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:51.056606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] RBAC: Authorizing `identity:get_role(role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:51.057967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:51.059374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 558/1113] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:50 2022] GET /identity/v3/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:51.065141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:51.152562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:51.163128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360232c0-bdd9-477b-b859-f3d993b5c64a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:51.195857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:51.196717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:51.196997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:51.197268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] PATH_INFO: `/v3/users/40df30f297934c5997b3dbc117b371f4` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:51.209834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] RBAC: Authorizing `identity:get_user(user_id=40df30f297934c5997b3dbc117b371f4)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:51.211315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:51.213072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 556/1114] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:51 2022] GET /identity/v3/users/40df30f297934c5997b3dbc117b371f4 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:51.218871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:51.287872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:51.316536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11aee76-a367-408c-9e1c-acebe5328851 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:51.349148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:51.349781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:51.350679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:51.350958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:51.415936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:51.417373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:51.481998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:51.511611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 559/1115] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:51.517564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:51.614333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8fa50-42ff-4970-8b67-b854991a5b1c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:51.647468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1303b-5571-433f-8c69-3145bb94f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:51.648317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1303b-5571-433f-8c69-3145bb94f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:51.648601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1303b-5571-433f-8c69-3145bb94f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:51.648874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1303b-5571-433f-8c69-3145bb94f3a5 None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:51.650051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1303b-5571-433f-8c69-3145bb94f3a5 None admin] RBAC: Authorizing `identity:get_project(project_id=fee14ca70d154433bcb19b8a55bf516e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:51.651516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1303b-5571-433f-8c69-3145bb94f3a5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:51.653182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 557/1116] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:40:51 2022] GET /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:51.659165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:51.754506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbee83e-47da-43b2-bcb0-b583d9ce9543 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:51.788462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:51.789092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:51.789367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:51.789636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:51.790453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] RBAC: Adding query filter params (role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=fee14ca70d154433bcb19b8a55bf516e, user.id=40df30f297934c5997b3dbc117b371f4) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:51.790761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:51.792199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8405f729-bca8-4c3b-a920-43d954701010 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:51.797449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 560/1117] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:51 2022] GET /identity/v3/role_assignments?user.id=40df30f297934c5997b3dbc117b371f4&scope.project.id=fee14ca70d154433bcb19b8a55bf516e&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:53.810314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d736-14d6-4eaf-8b2a-f7fb8aab936d None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:53.810887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d736-14d6-4eaf-8b2a-f7fb8aab936d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:53.810887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d736-14d6-4eaf-8b2a-f7fb8aab936d None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:53.812268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 558/1118] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:53.817475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:53.817912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:53.818018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:53.835807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:53.836993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:53.865165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:53.904458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 561/1119] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:53.926963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d736-14d6-4eaf-8b2a-f7fb8aab936d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:53.945757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:54.026078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d736-14d6-4eaf-8b2a-f7fb8aab936d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:54.059296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:54.060147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:54.060479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:54.060702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] PATH_INFO: `/v3/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:54.062164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] RBAC: Authorizing `identity:get_role(role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:54.063604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:54.064935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 559/1120] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:53 2022] GET /identity/v3/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:54.070838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:54.167856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd32db2-1329-4183-9199-a6f211cceda9 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:54.200799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:54.201655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:54.201937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:54.202209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] PATH_INFO: `/v3/users/40df30f297934c5997b3dbc117b371f4` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:54.203426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] RBAC: Authorizing `identity:get_user(user_id=40df30f297934c5997b3dbc117b371f4)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:54.204853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:54.206486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 562/1121] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:54 2022] GET /identity/v3/users/40df30f297934c5997b3dbc117b371f4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:54.212457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:54.294905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:54.311030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918df7f-3208-49a6-8582-cc5abf998963 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:54.344009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c825c26-2e60-475f-b7b0-c17735f4ec3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:54.344877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c825c26-2e60-475f-b7b0-c17735f4ec3a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:54.345159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c825c26-2e60-475f-b7b0-c17735f4ec3a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:54.345438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c825c26-2e60-475f-b7b0-c17735f4ec3a None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:54.346608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c825c26-2e60-475f-b7b0-c17735f4ec3a None admin] RBAC: Authorizing `identity:get_project(project_id=fee14ca70d154433bcb19b8a55bf516e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:54.348072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c825c26-2e60-475f-b7b0-c17735f4ec3a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:54.349789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 560/1122] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:40:54 2022] GET /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:54.356749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:54.390334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:54.455220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684910b-5d39-4184-8eaf-a5f3786fba48 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:54.489061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d12c10-ce39-49e6-8dca-2f01be8a4820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:54.489938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d12c10-ce39-49e6-8dca-2f01be8a4820 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:54.490214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d12c10-ce39-49e6-8dca-2f01be8a4820 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:54.490527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d12c10-ce39-49e6-8dca-2f01be8a4820 None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e/users/40df30f297934c5997b3dbc117b371f4/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:54.492901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d12c10-ce39-49e6-8dca-2f01be8a4820 None admin] RBAC: Authorizing `identity:create_grant(project_id=fee14ca70d154433bcb19b8a55bf516e, user_id=40df30f297934c5997b3dbc117b371f4, role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:54.494326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d12c10-ce39-49e6-8dca-2f01be8a4820 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:54.506352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 563/1123] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:40:54 2022] PUT /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e/users/40df30f297934c5997b3dbc117b371f4/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:40:56.509490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad05e1-83c2-4c3e-9e1c-1ab8b97b5936 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:56.510013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad05e1-83c2-4c3e-9e1c-1ab8b97b5936 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:56.510111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad05e1-83c2-4c3e-9e1c-1ab8b97b5936 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:56.511437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 561/1124] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:56.516734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:56.517164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:56.517282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:56.533461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:56.534631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:56.599759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 564/1125] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:56.624253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad05e1-83c2-4c3e-9e1c-1ab8b97b5936 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:56.728066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad05e1-83c2-4c3e-9e1c-1ab8b97b5936 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:56.760710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:56.761501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:56.761784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:56.762062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] PATH_INFO: `/v3/roles/ea47b285b0f845c592e517ef0a65ac87` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:56.763565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] RBAC: Authorizing `identity:get_role(role_id=ea47b285b0f845c592e517ef0a65ac87)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:56.764935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:56.766293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 562/1126] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:56 2022] GET /identity/v3/roles/ea47b285b0f845c592e517ef0a65ac87 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:56.772823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:56.797069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:56.869450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcdac5-7247-43c8-bb97-fc3a60809e53 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:56.902049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:56.902900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:56.903214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:56.903491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] PATH_INFO: `/v3/users/40df30f297934c5997b3dbc117b371f4` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:56.904668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] RBAC: Authorizing `identity:get_user(user_id=40df30f297934c5997b3dbc117b371f4)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:56.906079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:56.907717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 565/1127] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:40:56 2022] GET /identity/v3/users/40df30f297934c5997b3dbc117b371f4 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:56.914031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:56.935046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:57.011181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3728cf-6902-471d-9055-f7fe4a93c353 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:57.044702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:57.045338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:57.045623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:57.045898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:57.111722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:57.113171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:57.133096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:57.208550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 563/1128] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:40:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:40:57.215322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:57.303629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:57.312937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9e8ff-5f6b-4bcd-88d4-468f6205d77a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:57.346509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17572d99-bf20-46c2-9a36-347f34eff649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:57.347378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572d99-bf20-46c2-9a36-347f34eff649 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:57.347661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572d99-bf20-46c2-9a36-347f34eff649 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:57.347936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17572d99-bf20-46c2-9a36-347f34eff649 None admin] PATH_INFO: `/v3/projects/fee14ca70d154433bcb19b8a55bf516e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:57.349113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17572d99-bf20-46c2-9a36-347f34eff649 None admin] RBAC: Authorizing `identity:get_project(project_id=fee14ca70d154433bcb19b8a55bf516e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:57.350765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17572d99-bf20-46c2-9a36-347f34eff649 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:57.352494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 566/1129] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:40:57 2022] GET /identity/v3/projects/fee14ca70d154433bcb19b8a55bf516e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:57.358699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:57.453917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd581e-6a08-437a-a769-a6ec7048b9e7 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:57.472797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:57.486825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:57.487500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:57.487769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:57.488045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:57.488858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] RBAC: Adding query filter params (role.id=ea47b285b0f845c592e517ef0a65ac87, scope.project.id=fee14ca70d154433bcb19b8a55bf516e, user.id=40df30f297934c5997b3dbc117b371f4) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:40:57.489163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:57.491228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b421266-7072-4f0e-b900-704a754b6fe3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:57.496832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 564/1130] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:40:57 2022] GET /identity/v3/role_assignments?user.id=40df30f297934c5997b3dbc117b371f4&scope.project.id=fee14ca70d154433bcb19b8a55bf516e&role.id=ea47b285b0f845c592e517ef0a65ac87 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:59.510797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443f9d4-e89b-41e3-b3a6-fe2e857b842e None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:59.511365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443f9d4-e89b-41e3-b3a6-fe2e857b842e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:59.511365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443f9d4-e89b-41e3-b3a6-fe2e857b842e None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:59.512766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 567/1131] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:40:59.517808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:59.518233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:59.518366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:59.535634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:40:59.536814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:40:59.600795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 565/1132] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:40:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:40:59.624369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443f9d4-e89b-41e3-b3a6-fe2e857b842e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:59.723898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443f9d4-e89b-41e3-b3a6-fe2e857b842e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:59.757716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b2eca-eafa-4b93-b0bf-b410416ec802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:59.758495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b2eca-eafa-4b93-b0bf-b410416ec802 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:59.758776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b2eca-eafa-4b93-b0bf-b410416ec802 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:59.759050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b2eca-eafa-4b93-b0bf-b410416ec802 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:59.760242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b2eca-eafa-4b93-b0bf-b410416ec802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:59.761712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b2eca-eafa-4b93-b0bf-b410416ec802 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:59.763314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 568/1133] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:40:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:40:59.769109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:40:59.864356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d0353-7e79-4274-83f3-a903c80a1423 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:40:59.878159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:40:59.898498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc5348-0866-466c-8f17-7052ca572692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:40:59.899180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5348-0866-466c-8f17-7052ca572692 None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:40:59.899463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5348-0866-466c-8f17-7052ca572692 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:40:59.899742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5348-0866-466c-8f17-7052ca572692 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:40:59.900452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5348-0866-466c-8f17-7052ca572692 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:40:59.901903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc5348-0866-466c-8f17-7052ca572692 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:40:59.925212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 566/1134] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:40:59 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:41:01.954964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327253-39be-4049-b54d-541353bebdb3 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:01.955528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327253-39be-4049-b54d-541353bebdb3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:01.955528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de327253-39be-4049-b54d-541353bebdb3 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:01.956917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 569/1135] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:01.962340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:01.962779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:01.962898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:01.979358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:01.980552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:02.045950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 567/1136] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:02.071254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327253-39be-4049-b54d-541353bebdb3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:02.099098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:02.171788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de327253-39be-4049-b54d-541353bebdb3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:02.205361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae2f46-d8ba-4375-9c14-c62994dc4d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:02.206078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae2f46-d8ba-4375-9c14-c62994dc4d7a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:02.206362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae2f46-d8ba-4375-9c14-c62994dc4d7a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:02.206640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae2f46-d8ba-4375-9c14-c62994dc4d7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:02.207833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae2f46-d8ba-4375-9c14-c62994dc4d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:02.209463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae2f46-d8ba-4375-9c14-c62994dc4d7a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:02.211103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 570/1137] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:41:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:02.216940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:02.311311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:02.317520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f0db9-8f0c-4bef-b29f-7526789ff87a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:02.351275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca6bc1-a23d-438a-a060-d74bb5c5b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:02.351931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6bc1-a23d-438a-a060-d74bb5c5b07a None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:02.352216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6bc1-a23d-438a-a060-d74bb5c5b07a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:02.352488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca6bc1-a23d-438a-a060-d74bb5c5b07a None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:02.353187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca6bc1-a23d-438a-a060-d74bb5c5b07a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:02.354618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca6bc1-a23d-438a-a060-d74bb5c5b07a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:02.373932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 568/1138] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:41:02 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:41:04.434930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67c63a-8210-4f2e-8989-2647804273b5 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:04.435507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67c63a-8210-4f2e-8989-2647804273b5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:04.435507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67c63a-8210-4f2e-8989-2647804273b5 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:04.436800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 571/1139] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:04.442146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:04.442560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:04.442696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:04.459652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:04.460837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:04.526190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 569/1140] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:04.549015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67c63a-8210-4f2e-8989-2647804273b5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:04.644403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:04.652710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67c63a-8210-4f2e-8989-2647804273b5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:04.686376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:04.687121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:04.687405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:04.687679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:04.698455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:41:04.706053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:04.707202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:41:04.708120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 572/1141] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:04.714000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:04.792643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:04.810985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ba8b5-38dc-4a89-b945-4e54d10a58e5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:04.844573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:04.845236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:04.845521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:04.845796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:04.846547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:04.846883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:04.848394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:04.856776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 570/1142] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:04.862794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:04.961120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd8f12-bd64-49c2-ad3e-963c67f32413 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:04.993467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:04.994309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:04.994589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:04.994862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] PATH_INFO: `/v3/users/7e4220bf87e24f73aa98d5c9f7477491` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:05.008279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] RBAC: Authorizing `identity:get_user(user_id=7e4220bf87e24f73aa98d5c9f7477491)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:05.009705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:05.011362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 573/1143] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:41:04 2022] GET /identity/v3/users/7e4220bf87e24f73aa98d5c9f7477491 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:05.016948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:05.114170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09664f8-da6f-4dc9-9aee-6911a5354df1 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:05.147032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:05.147772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:05.148058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:05.148338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:05.162446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:05.214324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:05.215733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:05.309520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 571/1144] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:41:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:41:05.315452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:05.330707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:05.412936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba3fc19-30ad-4f6c-beef-36cdba1f6dc3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:05.445257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad320890-b81c-4855-9f47-81e643d0b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:05.446095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad320890-b81c-4855-9f47-81e643d0b66d None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:05.446368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad320890-b81c-4855-9f47-81e643d0b66d None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:05.446636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad320890-b81c-4855-9f47-81e643d0b66d None admin] PATH_INFO: `/v3/projects/c82ce37324eb43daae061bc304d06016` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:05.447821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad320890-b81c-4855-9f47-81e643d0b66d None admin] RBAC: Authorizing `identity:get_project(project_id=c82ce37324eb43daae061bc304d06016)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:05.449234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad320890-b81c-4855-9f47-81e643d0b66d None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:05.450908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 574/1145] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:41:05 2022] GET /identity/v3/projects/c82ce37324eb43daae061bc304d06016 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:05.456938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:05.500355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:05.552062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2d32-48dc-45b0-88d2-0d50814cdc9f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:05.585550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:05.586189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:05.586466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:05.586738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:05.587592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=c82ce37324eb43daae061bc304d06016, user.id=7e4220bf87e24f73aa98d5c9f7477491) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:05.587897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:05.589319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a0f3b-2272-4466-832d-3b4c300aeb90 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:05.594527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 572/1146] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:41:05 2022] GET /identity/v3/role_assignments?user.id=7e4220bf87e24f73aa98d5c9f7477491&scope.project.id=c82ce37324eb43daae061bc304d06016&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:07.594039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6b5f-9605-4e95-a9e3-d0bf903f2094 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:07.594570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6b5f-9605-4e95-a9e3-d0bf903f2094 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:07.594570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa6b5f-9605-4e95-a9e3-d0bf903f2094 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:07.595998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 575/1147] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:07.601464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e26662-3405-4794-9213-1b315da08f8e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:07.601880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e26662-3405-4794-9213-1b315da08f8e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:07.602023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e26662-3405-4794-9213-1b315da08f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:07.618775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-03e26662-3405-4794-9213-1b315da08f8e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:07.620000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-03e26662-3405-4794-9213-1b315da08f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:07.684851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 573/1148] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:07.708560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa6b5f-9605-4e95-a9e3-d0bf903f2094 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:07.788920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:07.808210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa6b5f-9605-4e95-a9e3-d0bf903f2094 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:07.841533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:07.842302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:07.842577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:07.842846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:07.852707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:41:07.860186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:07.861560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:41:07.862611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 576/1149] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:07.868355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e26662-3405-4794-9213-1b315da08f8e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:07.907193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:07.964993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e26662-3405-4794-9213-1b315da08f8e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:07.998833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:07.999523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:07.999807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:08.000085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:08.000833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:08.001140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:08.002508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:08.010931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 574/1150] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:08.016771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:08.114651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b27ffa-6628-4c7b-939e-fc5ba90b578b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:08.147555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:08.148410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:08.148689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:08.148958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] PATH_INFO: `/v3/users/7e4220bf87e24f73aa98d5c9f7477491` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:08.150125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] RBAC: Authorizing `identity:get_user(user_id=7e4220bf87e24f73aa98d5c9f7477491)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:08.151572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:08.153192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 577/1151] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:41:08 2022] GET /identity/v3/users/7e4220bf87e24f73aa98d5c9f7477491 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:08.158992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:08.254693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:08.257787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625d299-108c-4483-baf8-cb6b7dfedb4c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:08.291468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c1421-b261-4ee8-876e-1038dd58107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:08.292356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c1421-b261-4ee8-876e-1038dd58107c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:08.292640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c1421-b261-4ee8-876e-1038dd58107c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:08.292917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c1421-b261-4ee8-876e-1038dd58107c None admin] PATH_INFO: `/v3/projects/c82ce37324eb43daae061bc304d06016` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:08.294096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c1421-b261-4ee8-876e-1038dd58107c None admin] RBAC: Authorizing `identity:get_project(project_id=c82ce37324eb43daae061bc304d06016)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:08.295573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c1421-b261-4ee8-876e-1038dd58107c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:08.297271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 575/1152] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:41:08 2022] GET /identity/v3/projects/c82ce37324eb43daae061bc304d06016 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:08.303956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:08.316506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:08.402501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a8ebde-c291-427b-8442-b718a39c005e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:08.435805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4a0ca-fcd0-47a4-92f9-a195dcb4fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:08.436682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4a0ca-fcd0-47a4-92f9-a195dcb4fb54 None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:08.436967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4a0ca-fcd0-47a4-92f9-a195dcb4fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:08.437247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4a0ca-fcd0-47a4-92f9-a195dcb4fb54 None admin] PATH_INFO: `/v3/projects/c82ce37324eb43daae061bc304d06016/users/7e4220bf87e24f73aa98d5c9f7477491/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:08.439234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4a0ca-fcd0-47a4-92f9-a195dcb4fb54 None admin] RBAC: Authorizing `identity:create_grant(project_id=c82ce37324eb43daae061bc304d06016, user_id=7e4220bf87e24f73aa98d5c9f7477491, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:08.440663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4a0ca-fcd0-47a4-92f9-a195dcb4fb54 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:08.453473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 578/1153] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:41:08 2022] PUT /identity/v3/projects/c82ce37324eb43daae061bc304d06016/users/7e4220bf87e24f73aa98d5c9f7477491/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:41:10.435794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd56a6-b778-433f-b6e2-57408f416ed4 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:10.436302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd56a6-b778-433f-b6e2-57408f416ed4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:10.436302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd56a6-b778-433f-b6e2-57408f416ed4 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:10.437630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 576/1154] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:10.442656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:10.443095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:10.443249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:10.460400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:10.461589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:10.523396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:10.528607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 579/1155] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:10.551391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd56a6-b778-433f-b6e2-57408f416ed4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:10.651582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd56a6-b778-433f-b6e2-57408f416ed4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:10.683511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:10.684299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:10.684582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:10.684855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:10.695847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:41:10.703479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:10.704606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:41:10.705817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 577/1156] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:10.711364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:10.810595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de426e4-6a77-4e4a-9966-d380aa9aa199 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:10.844575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:10.845230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:10.845513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:10.845789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:10.846536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:10.846856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:10.848233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:10.856244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 580/1157] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:10.862282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:10.891122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:10.960500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7aeec-52ce-415c-b47c-01d024bf0577 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:10.993255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:10.994106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:10.994386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:10.994655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] PATH_INFO: `/v3/users/7e4220bf87e24f73aa98d5c9f7477491` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:10.995981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] RBAC: Authorizing `identity:get_user(user_id=7e4220bf87e24f73aa98d5c9f7477491)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:10.997401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:10.998997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 578/1158] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:41:10 2022] GET /identity/v3/users/7e4220bf87e24f73aa98d5c9f7477491 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:11.004672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:11.064736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:11.103139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e040d6-d63b-4cb7-968f-89931e14c361 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:11.136126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:11.136759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:11.137038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:11.137315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:11.201952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:11.203409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:11.260684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:11.300224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 581/1159] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:41:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:41:11.306608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:11.396739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:11.402844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e83aaa-14d9-4a48-8521-2f7ce29a1091 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:11.435052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093e1f8-0e79-4d73-a260-eae9aeede339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:11.435907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e1f8-0e79-4d73-a260-eae9aeede339 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:11.436437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e1f8-0e79-4d73-a260-eae9aeede339 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:11.436437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093e1f8-0e79-4d73-a260-eae9aeede339 None admin] PATH_INFO: `/v3/projects/c82ce37324eb43daae061bc304d06016` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:11.437606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093e1f8-0e79-4d73-a260-eae9aeede339 None admin] RBAC: Authorizing `identity:get_project(project_id=c82ce37324eb43daae061bc304d06016)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:11.439025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093e1f8-0e79-4d73-a260-eae9aeede339 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:11.440705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 579/1160] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:41:11 2022] GET /identity/v3/projects/c82ce37324eb43daae061bc304d06016 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:11.447306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:11.544219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d2b89-45dd-4f80-aec1-936d36d95b86 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:11.577235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:11.577875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:11.578157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:11.578432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:11.579286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=c82ce37324eb43daae061bc304d06016, user.id=7e4220bf87e24f73aa98d5c9f7477491) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:11.579602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:11.581021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346c2b3-431b-46fa-9722-43364a3ccc8b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:11.586916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 582/1161] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:41:11 2022] GET /identity/v3/role_assignments?user.id=7e4220bf87e24f73aa98d5c9f7477491&scope.project.id=c82ce37324eb43daae061bc304d06016&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:13.637423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9635a9-10c0-4308-9bf2-07c257f78bc2 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:13.637962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9635a9-10c0-4308-9bf2-07c257f78bc2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:13.637962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9635a9-10c0-4308-9bf2-07c257f78bc2 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:13.639643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 580/1162] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:13.645007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:13.645431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:13.645553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:13.662447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:13.663669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:13.683572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:13.728776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 583/1163] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:13.751767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9635a9-10c0-4308-9bf2-07c257f78bc2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:13.850360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9635a9-10c0-4308-9bf2-07c257f78bc2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:13.882743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:13.883403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:13.883687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:13.883964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:13.914155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:13.949558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:13.951002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:14.045754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 581/1164] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:41:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:41:14.051790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:14.148538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb221c-1377-4807-b71f-50f78877c35b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:14.181922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:14.182641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:14.182921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:14.183221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:14.189726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:14.191182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-965ff5f3-98cc-4ec0-a441-852f3e153e9f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:14.205056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:14.206241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:41:14.207501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 584/1165] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:41:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:14.212860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:14.274818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:14.313506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e121024-0255-460e-a876-e8a3343c9f77 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:14.347286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:14.347939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:14.348223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:14.348505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:14.349229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:14.349661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:14.351030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b13442-496c-4ee7-a821-62472c8c0299 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:14.358064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 582/1166] 10.222.0.86 () {56 vars in 1115 bytes} [Wed Jul 20 21:41:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:16.334397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59938671-581c-45e6-b977-977375a08174 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:16.334897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59938671-581c-45e6-b977-977375a08174 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:16.334897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59938671-581c-45e6-b977-977375a08174 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:16.336283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 585/1167] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:16.341526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5827f-65d5-492d-80fb-9f44991cbf1d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:16.341903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5827f-65d5-492d-80fb-9f44991cbf1d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:16.342087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5827f-65d5-492d-80fb-9f44991cbf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:16.358588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b6d5827f-65d5-492d-80fb-9f44991cbf1d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:16.359809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b6d5827f-65d5-492d-80fb-9f44991cbf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:16.425087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 583/1168] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:16.448850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59938671-581c-45e6-b977-977375a08174 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:16.467383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:16.550006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59938671-581c-45e6-b977-977375a08174 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:16.582584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db00dc-f392-4763-8c1f-f1e09c8c155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:16.583262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db00dc-f392-4763-8c1f-f1e09c8c155e None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:16.583548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db00dc-f392-4763-8c1f-f1e09c8c155e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:16.583825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db00dc-f392-4763-8c1f-f1e09c8c155e None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:16.584730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db00dc-f392-4763-8c1f-f1e09c8c155e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:16.586040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db00dc-f392-4763-8c1f-f1e09c8c155e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:16.600939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 586/1169] 10.222.0.86 () {60 vars in 1138 bytes} [Wed Jul 20 21:41:16 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:41:18.634646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53416deb-f594-438f-96f1-0c494c486129 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:18.635388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53416deb-f594-438f-96f1-0c494c486129 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:18.635388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53416deb-f594-438f-96f1-0c494c486129 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:18.636592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 584/1170] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:18.641719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:18.642124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:18.642271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:18.658653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:18.659868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:18.723894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 587/1171] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:18.748167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53416deb-f594-438f-96f1-0c494c486129 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:18.807479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:18.847732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53416deb-f594-438f-96f1-0c494c486129 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:18.881432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc5a228-e180-4fea-a09c-41530205f6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:18.882271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5a228-e180-4fea-a09c-41530205f6cc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:18.882548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5a228-e180-4fea-a09c-41530205f6cc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:18.882819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc5a228-e180-4fea-a09c-41530205f6cc None admin] PATH_INFO: `/v3/domains/4e4b555b6b4d4cc3bba1ae99e3c889c4` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:18.893271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5a228-e180-4fea-a09c-41530205f6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=4e4b555b6b4d4cc3bba1ae99e3c889c4)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:18.894668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc5a228-e180-4fea-a09c-41530205f6cc None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:18.896421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 585/1172] 10.222.0.86 () {56 vars in 1183 bytes} [Wed Jul 20 21:41:18 2022] GET /identity/v3/domains/4e4b555b6b4d4cc3bba1ae99e3c889c4 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:18.902522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:18.910648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:19.000711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c670123-45fa-4e52-9088-cbd75cab426a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:19.033069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c81275-f7eb-47f4-a76d-80b285226dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:19.033719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c81275-f7eb-47f4-a76d-80b285226dff None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:19.034007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c81275-f7eb-47f4-a76d-80b285226dff None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:19.034278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c81275-f7eb-47f4-a76d-80b285226dff None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:19.035000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c81275-f7eb-47f4-a76d-80b285226dff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:19.036438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c81275-f7eb-47f4-a76d-80b285226dff None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:19.061549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 588/1173] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:41:18 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:41:21.122126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ca8f-2041-42e8-a6c8-81dba5473af7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:21.122656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ca8f-2041-42e8-a6c8-81dba5473af7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:21.122656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e4ca8f-2041-42e8-a6c8-81dba5473af7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:21.124077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 586/1174] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:21.129419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:21.129835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:21.129972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:21.146378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:21.147677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:21.199645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:21.214040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 589/1175] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:21.239428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4ca8f-2041-42e8-a6c8-81dba5473af7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:21.339371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e4ca8f-2041-42e8-a6c8-81dba5473af7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:21.356119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:21.373283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3f38f-2250-4d64-871d-43b704d42641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:21.374138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3f38f-2250-4d64-871d-43b704d42641 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:21.374419 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3f38f-2250-4d64-871d-43b704d42641 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:21.374693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3f38f-2250-4d64-871d-43b704d42641 None admin] PATH_INFO: `/v3/domains/4e4b555b6b4d4cc3bba1ae99e3c889c4` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:21.375905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3f38f-2250-4d64-871d-43b704d42641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4e4b555b6b4d4cc3bba1ae99e3c889c4)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:21.377340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3f38f-2250-4d64-871d-43b704d42641 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:21.378948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 587/1176] 10.222.0.86 () {56 vars in 1183 bytes} [Wed Jul 20 21:41:21 2022] GET /identity/v3/domains/4e4b555b6b4d4cc3bba1ae99e3c889c4 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:21.384815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:21.481218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82c8b5-6d6d-49aa-ba46-b0e9b1970b25 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:21.514172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d53a11-7380-4df7-bc0d-28aba55d21d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:21.514960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53a11-7380-4df7-bc0d-28aba55d21d1 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:21.515276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53a11-7380-4df7-bc0d-28aba55d21d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:21.515553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d53a11-7380-4df7-bc0d-28aba55d21d1 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:21.516274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53a11-7380-4df7-bc0d-28aba55d21d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:21.517693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d53a11-7380-4df7-bc0d-28aba55d21d1 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:21.537180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 590/1177] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:41:21 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:41:23.593722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f5c2-933b-4008-89d5-cc48e62b7e86 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:23.594305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f5c2-933b-4008-89d5-cc48e62b7e86 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:23.594305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90f5c2-933b-4008-89d5-cc48e62b7e86 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:23.595648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 588/1178] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:23.600774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:23.601165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:23.601331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:23.618756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:23.619982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:23.659343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:23.685495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 591/1179] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:23.709615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90f5c2-933b-4008-89d5-cc48e62b7e86 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:23.809666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90f5c2-933b-4008-89d5-cc48e62b7e86 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:23.842435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:23.843243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:23.843527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:23.843852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:23.854152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:41:23.861971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:23.863206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:41:23.864303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 589/1180] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:23.869600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:23.965642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0419769-9747-4556-ba53-9c2d2b26326f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:24.000885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:24.001544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:24.001823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:24.002094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:24.002837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:24.003171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:24.004569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:24.012371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 592/1181] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:24.018179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:24.064455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:24.117966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb222a-5484-43cf-9e11-f80ea2bf3c48 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:24.150571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:24.151464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:24.151745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:24.152017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] PATH_INFO: `/v3/users/d9aee7100e30417291bbd0f09201e07e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:24.165175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] RBAC: Authorizing `identity:get_user(user_id=d9aee7100e30417291bbd0f09201e07e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:24.166601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:24.168245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 590/1182] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:41:24 2022] GET /identity/v3/users/d9aee7100e30417291bbd0f09201e07e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:24.174018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:24.209015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:24.274193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675a7d8-3cfc-45c8-ad25-fc264290e0a9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:24.308150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:24.308796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:24.309082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:24.309359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:24.375422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:24.376790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:24.408662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:24.473081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 593/1183] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:41:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:41:24.479547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:24.576104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2f53-7583-4dbb-8807-6b0828954d10 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:24.578369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:24.609738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e8979-be55-4523-9725-1d3aa91db377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:24.610585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e8979-be55-4523-9725-1d3aa91db377 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:24.610869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e8979-be55-4523-9725-1d3aa91db377 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:24.611173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e8979-be55-4523-9725-1d3aa91db377 None admin] PATH_INFO: `/v3/projects/c47254754b2749d08fbc1b103f78652a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:24.612606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e8979-be55-4523-9725-1d3aa91db377 None admin] RBAC: Authorizing `identity:get_project(project_id=c47254754b2749d08fbc1b103f78652a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:24.614038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e8979-be55-4523-9725-1d3aa91db377 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:24.615748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 591/1184] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:41:24 2022] GET /identity/v3/projects/c47254754b2749d08fbc1b103f78652a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:24.621981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:24.719253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a5d00-d4d4-412e-afc5-58441d1943e2 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:24.752287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:24.752927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:24.753207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:24.753488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:24.754306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=c47254754b2749d08fbc1b103f78652a, user.id=d9aee7100e30417291bbd0f09201e07e) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:24.754616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:24.756185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dcb84-2696-453d-953b-198e0dd01b65 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:24.759553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:24.762434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 594/1185] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:41:24 2022] GET /identity/v3/role_assignments?user.id=d9aee7100e30417291bbd0f09201e07e&scope.project.id=c47254754b2749d08fbc1b103f78652a&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:26.763259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d649418-7e32-4b13-a189-689443c4bd13 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:26.763796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d649418-7e32-4b13-a189-689443c4bd13 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:26.763796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d649418-7e32-4b13-a189-689443c4bd13 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:26.765172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 592/1186] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:26.770288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:26.770681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:26.770834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:26.789233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:26.790443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:26.854030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 595/1187] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:41:26.876470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d649418-7e32-4b13-a189-689443c4bd13 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:26.977093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d649418-7e32-4b13-a189-689443c4bd13 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:27.010861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:27.011668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:27.011992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:27.012267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:27.022132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:41:27.030374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:27.031521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:41:27.032427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 593/1188] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:41:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:27.037715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:27.137316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176f8f9-03e5-4a68-8f31-823ff4519a54 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:27.166254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:27.172333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:27.173010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:27.173290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:27.173568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:27.174318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:27.174629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:27.176027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:27.184097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 596/1189] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:27.189910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:27.221571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:27.289029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa412c40-0073-4c70-9275-f10a35a3de5c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:27.321966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:27.322815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:27.323121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:27.323399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] PATH_INFO: `/v3/users/d9aee7100e30417291bbd0f09201e07e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:27.324578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] RBAC: Authorizing `identity:get_user(user_id=d9aee7100e30417291bbd0f09201e07e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:27.326016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:27.327652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 594/1190] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:41:27 2022] GET /identity/v3/users/d9aee7100e30417291bbd0f09201e07e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:27.333360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:27.431316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6bd68-a977-4c67-95a5-8472cbbccf7b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:27.464273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f2c49-911e-45a5-95cf-ad96cd929e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:27.465114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f2c49-911e-45a5-95cf-ad96cd929e8a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:27.465397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f2c49-911e-45a5-95cf-ad96cd929e8a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:27.465673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f2c49-911e-45a5-95cf-ad96cd929e8a None admin] PATH_INFO: `/v3/projects/c47254754b2749d08fbc1b103f78652a` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:27.466853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f2c49-911e-45a5-95cf-ad96cd929e8a None admin] RBAC: Authorizing `identity:get_project(project_id=c47254754b2749d08fbc1b103f78652a)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:27.468310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f2c49-911e-45a5-95cf-ad96cd929e8a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:27.470012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 597/1191] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:41:27 2022] GET /identity/v3/projects/c47254754b2749d08fbc1b103f78652a => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:27.475810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:27.568194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:27.572039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5712e1d5-37a5-4eb5-b75f-f8af4f9a4985 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:27.604222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b080e-efcd-4b79-a484-238969acd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:27.605077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b080e-efcd-4b79-a484-238969acd029 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:27.605354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b080e-efcd-4b79-a484-238969acd029 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:27.605620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b080e-efcd-4b79-a484-238969acd029 None admin] PATH_INFO: `/v3/projects/c47254754b2749d08fbc1b103f78652a/users/d9aee7100e30417291bbd0f09201e07e/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:27.607558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b080e-efcd-4b79-a484-238969acd029 None admin] RBAC: Authorizing `identity:create_grant(project_id=c47254754b2749d08fbc1b103f78652a, user_id=d9aee7100e30417291bbd0f09201e07e, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:27.608968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b080e-efcd-4b79-a484-238969acd029 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:27.621868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 595/1192] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:41:27 2022] PUT /identity/v3/projects/c47254754b2749d08fbc1b103f78652a/users/d9aee7100e30417291bbd0f09201e07e/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:41:29.625762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b201b4-8617-4b30-8a8d-976a2a301c8b None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:29.626317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b201b4-8617-4b30-8a8d-976a2a301c8b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:29.626317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b201b4-8617-4b30-8a8d-976a2a301c8b None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:29.627773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 598/1193] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:41:29.632891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:29.633331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:29.633427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:29.650156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:41:29.651368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:41:29.716289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 596/1194] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:41:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:41:29.739597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b201b4-8617-4b30-8a8d-976a2a301c8b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:29.798666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:29.839509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b201b4-8617-4b30-8a8d-976a2a301c8b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:29.871882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:29.872649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:29.872925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:29.873192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:29.883497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:41:29.890799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:41:29.891786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:41:29.892779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 599/1195] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:41:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:41:29.898584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:29.984894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:29.995254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e099e-b518-4502-880b-dd15fb207f7b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:30.029489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:30.030149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:30.030430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:30.030704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:30.031476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:30.031787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:30.033159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:30.041532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 597/1196] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:41:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:30.047335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:30.145266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452a09a-1530-49d8-be40-838497220a2b None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:30.164961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:30.178719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:30.179930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:30.180214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:30.180490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] PATH_INFO: `/v3/users/d9aee7100e30417291bbd0f09201e07e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:30.181671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] RBAC: Authorizing `identity:get_user(user_id=d9aee7100e30417291bbd0f09201e07e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:30.183119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:30.184745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 600/1197] 10.222.0.86 () {56 vars in 1177 bytes} [Wed Jul 20 21:41:30 2022] GET /identity/v3/users/d9aee7100e30417291bbd0f09201e07e => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:30.190487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:30.288989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057cd7d-5fa6-4ddc-97fc-1ba7418147e5 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:30.322671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:30.323332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:30.323619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:30.323899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:30.344135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:30.389626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:30.391094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:30.485287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 598/1198] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:41:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:41:30.492157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:30.587996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c6399-c22e-40b0-9e0c-87c293ed2452 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:30.620625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7987db-812b-4ab9-ba06-17e17914ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:30.621464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7987db-812b-4ab9-ba06-17e17914ab26 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:30.621743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7987db-812b-4ab9-ba06-17e17914ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:30.622013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7987db-812b-4ab9-ba06-17e17914ab26 None admin] PATH_INFO: `/v3/projects/c47254754b2749d08fbc1b103f78652a` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:30.623202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7987db-812b-4ab9-ba06-17e17914ab26 None admin] RBAC: Authorizing `identity:get_project(project_id=c47254754b2749d08fbc1b103f78652a)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:30.624617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7987db-812b-4ab9-ba06-17e17914ab26 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:30.626291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 601/1199] 10.222.0.86 () {56 vars in 1186 bytes} [Wed Jul 20 21:41:30 2022] GET /identity/v3/projects/c47254754b2749d08fbc1b103f78652a => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:30.633309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:41:30.684050 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:30.731741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe799-d0bb-4624-a764-cf08b1ec1dd3 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:41:30.766565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:41:30.767171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:41:30.767458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:41:30.767733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:41:30.768559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=c47254754b2749d08fbc1b103f78652a, user.id=d9aee7100e30417291bbd0f09201e07e) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:41:30.768872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:41:30.770297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e50eb-890e-449f-9acf-7e17ea8fb598 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:41:30.776175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 599/1200] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:41:30 2022] GET /identity/v3/role_assignments?user.id=d9aee7100e30417291bbd0f09201e07e&scope.project.id=c47254754b2749d08fbc1b103f78652a&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:41:40.616074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=72 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:41:40.773821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=42 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:26.055559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ad684-4a2f-4009-a6d7-2e27f07068a8 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:26.056168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ad684-4a2f-4009-a6d7-2e27f07068a8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:26.056168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ad684-4a2f-4009-a6d7-2e27f07068a8 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:26.057478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 602/1201] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:42:26.064941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:26.065489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:26.065489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:26.088878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:26.088878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:26.170462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 600/1202] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:26.200018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ad684-4a2f-4009-a6d7-2e27f07068a8 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:26.318260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ad684-4a2f-4009-a6d7-2e27f07068a8 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:26.364530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:26.365367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:26.365697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:26.365997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:26.377791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:26.379269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:26.403829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:26.405100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:26.406386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:26.406386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:26.406386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:26.406386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:26.410187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 603/1203] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:26.411602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:26.559688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a00fa-dd2d-4510-96c4-f1a2815bd564 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:26.597680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:26.607523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:26.608291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:26.608645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:26.608933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:26.609801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:26.610114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:26.611920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ab5ec-dd51-443c-8839-874961399aba None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:26.631294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 601/1204] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:26.632603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:26.711364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:26.780494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37ff79-d8e7-4bc8-8775-592912c2438a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:26.840078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b03a51-2fd5-4ab3-96ad-5d61053b18a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:26.840976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b03a51-2fd5-4ab3-96ad-5d61053b18a0 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:26.841392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b03a51-2fd5-4ab3-96ad-5d61053b18a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:26.841759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b03a51-2fd5-4ab3-96ad-5d61053b18a0 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:26.842687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b03a51-2fd5-4ab3-96ad-5d61053b18a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:26.844374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b03a51-2fd5-4ab3-96ad-5d61053b18a0 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:26.873238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 604/1205] 10.222.0.86 () {60 vars in 1132 bytes} [Wed Jul 20 21:42:26 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:42:29.216071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c3ef-3f5f-4c88-bfef-46cd1884a740 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:29.216740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c3ef-3f5f-4c88-bfef-46cd1884a740 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:29.216846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37c3ef-3f5f-4c88-bfef-46cd1884a740 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:29.218289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 602/1206] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:42:29.224133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:29.225278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:29.225278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:29.247245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:29.248623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:29.330630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 605/1207] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:29.355631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c3ef-3f5f-4c88-bfef-46cd1884a740 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:29.481652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37c3ef-3f5f-4c88-bfef-46cd1884a740 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:29.528084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:29.528869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:29.529150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:29.529422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:29.542444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:42:29.550998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:29.557732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:29.558631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 603/1208] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:29.559783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:29.605251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:29.678878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c23bf-68d2-4327-addb-3d776192c202 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:29.719147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:29.719818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:29.720079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:29.720359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:29.721251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:29.721454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:29.722807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:29.731886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 606/1209] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:42:29.737530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:29.804553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:29.865703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfd363-9c62-4672-96c6-1a807cb2032b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:29.910508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:29.911537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:29.911865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:29.912356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:29.929316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:29.931007 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:29.952657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:29.953851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:29.954914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:29.954914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:29.954914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:29.954914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:29.954914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 604/1210] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:29.961905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:30.108239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ad7f4-5d6b-4722-825d-1f5f041d5f46 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:30.111589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:30.157056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:30.157842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:30.158617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:30.159105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:30.160010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:30.160480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:30.162127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:30.177180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 607/1211] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:30.183708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:30.301723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c67fe-85f9-4e1c-af76-98a368065fc2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:30.304219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:30.346777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:30.347623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:30.350474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:30.350740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:30.358937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:30.360832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:42:30.379285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:30.380376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:30.382325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 605/1212] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:42:30.388290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:30.512554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd9f8b-411f-4276-b0dd-e5a9f547a2f8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:30.550074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:30.550750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:30.551038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:30.551346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:30.552189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:30.552498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:30.554882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:30.573015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 608/1213] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Jul 20 21:42:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:42:30.579788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:30.713388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c65c1b-b3f3-4575-967f-722f453d5488 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:30.766813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:30.767788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:30.768249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:30.768525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:30.813064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:30.856442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:30.858179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:30.973370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 606/1214] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:42:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 21:42:30.979957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:31.000059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:31.104076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ef807-df7b-49e8-aa0d-4cd13b6762bb None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:31.140287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:31.142420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:31.142705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:31.142975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:31.154841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:31.156593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:31.176820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:31.177861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:31.179329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:31.179329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:31.179329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:31.179329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:31.179329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 609/1215] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:42:31.185054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:31.256927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:31.300269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e944-70b4-40ae-92ec-776a314f6962 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:31.337739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:31.338531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:31.338633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:31.338905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:31.339703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:31.340164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:31.341341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:31.353581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 607/1216] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:31.359570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:31.435898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:31.473015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa324-72ed-4a0b-88a0-73049b610d46 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:31.515602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:31.516404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:31.516749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:31.516953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:31.527987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:31.529443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:42:31.543668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:31.544825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:31.546006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:31.549666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 610/1217] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:31.550716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:31.664509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb952e-e9ec-44ad-8e9c-09928f20bd41 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:31.693371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:31.707454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:31.708128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:31.708408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:31.708685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:31.709505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:31.709908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:31.711374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81701c5e-357e-4c57-a492-38de65c1fb0f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:31.724371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 608/1218] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:31.730738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:31.846490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e357ad3-dbf8-4aff-aad9-96ba24832a6c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:31.876425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:31.883853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:31.884520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:31.884801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:31.885112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:31.885892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=06938dbfe3df4bc28f065c82315e4308) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:31.886206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:31.887667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfec024-a5b7-4fb6-9360-f574289c2256 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:31.903106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 611/1219] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:42:31 2022] GET /identity/v3/role_assignments?user.id=06938dbfe3df4bc28f065c82315e4308&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:34.018184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2925d97-77e8-4800-919a-ce60795145b5 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.018722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2925d97-77e8-4800-919a-ce60795145b5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.018833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2925d97-77e8-4800-919a-ce60795145b5 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.020153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 609/1220] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:42:34.025958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.026402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.026523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.043691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:34.044899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:34.121226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 612/1221] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:34.144379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2925d97-77e8-4800-919a-ce60795145b5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:34.257321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2925d97-77e8-4800-919a-ce60795145b5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:34.295591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:34.296318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.296596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.296865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.307837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:42:34.315638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:34.316654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:34.317526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 610/1222] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:34.323247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:34.429150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec0a20-836c-402f-a3e7-4a0cb4a2bf78 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:34.455347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:34.466512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721003d1-0014-40fc-96db-a103727f141a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:34.467223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721003d1-0014-40fc-96db-a103727f141a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.467527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721003d1-0014-40fc-96db-a103727f141a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.467843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721003d1-0014-40fc-96db-a103727f141a None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.468538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721003d1-0014-40fc-96db-a103727f141a None admin] RBAC: Adding query filter params (name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:34.468849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721003d1-0014-40fc-96db-a103727f141a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:34.470220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721003d1-0014-40fc-96db-a103727f141a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:34.479384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 613/1223] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:42:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:34.485374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:34.529407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:34.589116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51384286-264c-43d5-8281-1ac1b8f3e9cc None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:34.624005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:34.624933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.625539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.626053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.636878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:34.638361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:34.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:34.657149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:34.664130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:34.664130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:34.664130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:34.664130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:34.664130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 611/1224] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:42:34.664495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721003d1-0014-40fc-96db-a103727f141a None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:34.767541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721003d1-0014-40fc-96db-a103727f141a None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:34.806102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:34.806781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.807013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.807421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.808053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:34.808362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:34.809744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:34.822461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 614/1225] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:34.828464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:34.936807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:34.945726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab86024-7117-4097-98e7-a70bd8a25fc8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:34.985120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:34.986059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:34.986440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:34.986828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] PATH_INFO: `/v3/users/placement` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:34.996369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:34.997821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:42:35.012724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:35.013979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:35.015203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:35.019895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:35.020316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 612/1226] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:35.071158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:35.137223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae335b-57dd-40c5-b7a2-473b455b517e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:35.188803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:35.189625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:35.189960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:35.190333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:35.191569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:35.192050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:35.193691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:35.211866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 615/1227] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Jul 20 21:42:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:42:35.217719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:35.326761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93657406-1d2f-4729-9580-309b5fa7600b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:35.361820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:35.362609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:35.362895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:35.363451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:35.375824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:35.377542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:35.396898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:35.398899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:35.400026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:35.400026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:35.400026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:35.400026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:35.400026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 613/1228] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:42:35.404524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:35.516227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007da091-cb97-481e-b30b-2e667baf3597 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:35.518754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:35.555723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:35.556489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:35.556767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:35.557109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:35.557947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:35.558290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:35.565527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:35.580984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 616/1229] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:35.586911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:35.596888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:35.697524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb61ff9-5cb1-4164-812b-aae6bba3a477 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:35.736732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:35.737529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:35.737814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:35.738345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:35.745239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:35.746689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:42:35.761360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:35.762491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:35.763492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 614/1230] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:35.768821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:35.886592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fe908-2f21-4c4d-a5f1-45286bd58931 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:35.923691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:35.924359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:35.924646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:35.924946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:35.925710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:35.926024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:35.927483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d1c08-4ee8-493b-922b-5028100247c7 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:35.939719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 617/1231] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:35.946508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:36.022212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:36.058626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca75066-05cf-45a1-add7-b885e9655fbe None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:36.096592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8612a5f-81d0-4974-8d00-28d93bb9a7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:36.097490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612a5f-81d0-4974-8d00-28d93bb9a7d3 None admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:36.097773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612a5f-81d0-4974-8d00-28d93bb9a7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:36.098048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612a5f-81d0-4974-8d00-28d93bb9a7d3 None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/06938dbfe3df4bc28f065c82315e4308/roles/cdc059ca09d14d34b88e65afd478790f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:36.113711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8612a5f-81d0-4974-8d00-28d93bb9a7d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=06938dbfe3df4bc28f065c82315e4308, role_id=cdc059ca09d14d34b88e65afd478790f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:36.115747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8612a5f-81d0-4974-8d00-28d93bb9a7d3 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:36.129244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 615/1232] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:42:35 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/06938dbfe3df4bc28f065c82315e4308/roles/cdc059ca09d14d34b88e65afd478790f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:42:38.302451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd7642-2995-4a94-900e-ecb6ea4d85f3 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:38.303041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd7642-2995-4a94-900e-ecb6ea4d85f3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:38.303190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd7642-2995-4a94-900e-ecb6ea4d85f3 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:38.304573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 618/1233] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:42:38.310540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:38.311030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:38.311181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:38.328270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:38.329574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:38.407362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 616/1234] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:42:38.432065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd7642-2995-4a94-900e-ecb6ea4d85f3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:38.457733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:38.540180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd7642-2995-4a94-900e-ecb6ea4d85f3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:38.575461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:38.576255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:38.576630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:38.576880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:38.590048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:42:38.597747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:38.598769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:38.600627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 619/1235] 10.222.0.86 () {56 vars in 1102 bytes} [Wed Jul 20 21:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:38.606005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:38.655497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:38.715426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1ab45-5000-4f23-a8a8-ca2861e77dad None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:38.752871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:38.753571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:38.753853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:38.754126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:38.754920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:38.755259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:38.757646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:38.766376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 617/1236] 10.222.0.86 () {56 vars in 1103 bytes} [Wed Jul 20 21:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:42:38.772970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:38.887168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:38.905925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990746-c4e8-4934-8ea9-fa0bf7a7f791 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:38.943123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:38.944517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:38.944799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:38.945069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:38.955888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:38.957336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:38.975005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:38.976354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:38.977302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:38.977302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:38.977302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:38.977302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:38.977302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 620/1237] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:38.982228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:39.097400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618a9b4-ef9f-49f1-8fe7-f50a166be3a6 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:39.107774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:39.136130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:39.136832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:39.137085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:39.137379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:39.138093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:39.138401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:39.139825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:39.151658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 618/1238] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:39.157451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:39.272369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d6483-f908-4e72-b36a-1927898c2869 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:39.312016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:39.313014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:39.313294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:39.313564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:39.316470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:39.323800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:39.325785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:42:39.340628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:39.341759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:39.343185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 621/1239] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:39.349403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:39.465291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ebbca-1f74-473c-aac0-92a06d8e5a2b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:39.509464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:39.510137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:39.510423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:39.510701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:39.511581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:39.511899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:39.513509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:39.532864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 619/1240] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Jul 20 21:42:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:39.540530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:39.671635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478efe-42d3-4f26-8dca-172c136a7ee8 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:39.709680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:39.710328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:39.710608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:39.710883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:39.784084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:39.792310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:39.793854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:39.903142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 622/1241] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:42:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:42:39.909424 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:39.939213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:40.019903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d24f9-4dd6-4ae0-a2b6-c112f3b7ba5a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:40.059806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:40.060696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:40.060977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:40.061248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:40.072534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:40.074231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:40.097919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:40.099120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:40.100902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:40.100902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:40.100902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:40.100902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:40.100902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 620/1242] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:40.105563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:40.206811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:40.214731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494b521a-15d8-475b-856e-276d0bcd8aa6 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:40.251488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:40.252159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:40.252440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:40.252712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:40.253441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:40.253807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:40.255208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:40.267477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 623/1243] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:42:40.274181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:40.369412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:40.394519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2328b96b-8337-4d6b-80ec-fef3b0b6281e None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:40.431488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:40.432289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:40.432573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:40.432850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:40.440264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:40.441990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:42:40.458670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:40.459824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:40.460880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 621/1244] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:42:40.466462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:40.581165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a225a3-bcff-491b-8fa8-fe02fcb6ef27 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:40.621881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:40.622634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:40.623118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:40.623423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:40.624406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:40.624818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:40.626602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a013997-762a-4663-9141-c7c8fc5b134a None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:40.642294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 624/1245] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:40.649224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:40.759869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af658e-2bef-455c-81ea-84b893e22438 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:40.796201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:40.796868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:40.797152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:40.797448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:40.798267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] RBAC: Adding query filter params (role.id=cdc059ca09d14d34b88e65afd478790f, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=06938dbfe3df4bc28f065c82315e4308) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:40.798583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:40.800056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd32d3d-c914-4635-aec6-1720438d7dab None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:40.808031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 622/1246] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:42:40 2022] GET /identity/v3/role_assignments?user.id=06938dbfe3df4bc28f065c82315e4308&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=cdc059ca09d14d34b88e65afd478790f => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:43.099297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c206f25-cbf2-4e3f-a7e7-01ee67169245 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:43.100492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c206f25-cbf2-4e3f-a7e7-01ee67169245 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:43.100492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c206f25-cbf2-4e3f-a7e7-01ee67169245 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:43.101330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 625/1247] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:42:43.106858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:43.107342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:43.107434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:43.113746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:43.128093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:43.129377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:43.206140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 623/1248] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:43.231539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c206f25-cbf2-4e3f-a7e7-01ee67169245 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:43.251172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:43.353582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c206f25-cbf2-4e3f-a7e7-01ee67169245 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:43.391001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:43.391839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:43.392123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:43.392396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:43.404777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:42:43.412532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:43.413719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:43.419708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 626/1249] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:42:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:43.420649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:43.527015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384bb58-271b-48e9-a4e2-f45f8b3468b1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:43.557688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:43.563346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:43.564011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:43.564287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:43.564554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:43.565378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:43.565683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:43.567197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:43.576267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 624/1250] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:42:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:43.582056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:43.681942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:43.702905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f115b8-1e39-415b-97da-1fb5d5cace60 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:43.743111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:43.743916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:43.744196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:43.744468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:43.755103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:43.757110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:43.773815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:43.774910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:43.776062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:43.776062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:43.776062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:43.776062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:43.776062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 627/1251] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:43.781325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:43.934836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d1bad-fb00-4f44-b0af-3664d2676572 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:43.981022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:43.981774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:43.982062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:43.982342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:43.983108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:43.983424 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:43.984839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:44.000501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 625/1252] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:44.006748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:44.139358 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03f275-6f70-45bf-8e52-aeec70dc973c None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:44.174175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:44.185096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:44.185827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:44.186106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:44.186376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:44.196915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:44.199241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:42:44.219007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:44.221387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:44.223272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 628/1253] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:44.227449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:44.294683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:44.361875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83453ec-60af-4233-ab82-2f97c7477d54 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:44.409232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:44.410934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:44.411394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:44.411724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:44.413688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:44.414074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:44.415598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:44.440559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 626/1254] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Jul 20 21:42:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:44.446631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:44.590361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b8280-2d4f-4da3-9dbc-8dcd673b5080 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:44.627737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:44.628360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:44.628643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:44.628924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:44.691199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:44.706956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:44.710686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:44.819250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 629/1255] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:42:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:42:44.825251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:44.943894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b8200-d996-4821-8fab-be653cadba9f None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:44.957309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:44.981407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:44.982209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:44.982489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:44.982761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:44.994966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:44.996475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:45.015750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:45.016904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:45.017834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:45.017834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:45.017834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:45.017834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:45.017834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 627/1256] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:45.023840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:45.123418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:45.137591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1711ed-5750-4e20-84f9-584f2bbba5ad None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:45.176358 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:45.177050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:45.177315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:45.177632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:45.178531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:45.178849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:45.180303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:45.193775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 630/1257] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:45.199759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:45.305222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1c32be-3ee5-461a-9075-68bc2c08a99b None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:45.341173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:45.342005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:45.342282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:45.342550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:45.350572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:45.352075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:42:45.366830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:45.368034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:45.369052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 628/1258] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:42:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:45.374176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:45.482551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335407f2-5f51-425f-8755-68fe8609a4b9 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:45.517847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831054e-5034-4617-b369-df51b48de751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:45.518516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831054e-5034-4617-b369-df51b48de751 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:45.518797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831054e-5034-4617-b369-df51b48de751 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:45.519093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831054e-5034-4617-b369-df51b48de751 None admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:45.519899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831054e-5034-4617-b369-df51b48de751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:45.520210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831054e-5034-4617-b369-df51b48de751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:45.522276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831054e-5034-4617-b369-df51b48de751 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:45.537134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 631/1259] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:45.543538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:45.569928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:45.650153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe1840-d83c-49e4-a885-6b1b5c5205a2 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:45.684504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:45.685163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:45.685446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:45.685720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:45.686557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=06938dbfe3df4bc28f065c82315e4308) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:45.686866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:45.688331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a2922-b89d-4451-93b0-c3ed567a8c60 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:45.695097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 629/1260] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:42:45 2022] GET /identity/v3/role_assignments?user.id=06938dbfe3df4bc28f065c82315e4308&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=62c943ce95d54a1999a447dca6a1103b => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:47.712818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf24d5-a388-42ca-98c5-b5bb6ac15ec9 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:47.713379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf24d5-a388-42ca-98c5-b5bb6ac15ec9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:47.713379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf24d5-a388-42ca-98c5-b5bb6ac15ec9 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:47.714755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 632/1261] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:42:47.719745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:47.720182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:47.720310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:47.736727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:47.737949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:47.806296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 630/1262] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:47.837837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf24d5-a388-42ca-98c5-b5bb6ac15ec9 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:47.850257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:47.943767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf24d5-a388-42ca-98c5-b5bb6ac15ec9 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:47.977422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:47.978204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:47.978488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:47.978765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:47.988935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:42:47.996518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:47.997614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:47.998648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 633/1263] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 21:42:48.003685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:48.022568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:48.102321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f732d9-a3c6-445a-ad89-d1690b40a158 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:48.136736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:48.137395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:48.137681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:48.137956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:48.138702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:48.139010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:48.140407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:48.148278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 631/1264] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:48.154060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:48.218627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:48.252176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21858513-4857-407b-ac1f-49ad743df31e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:48.284515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:48.285293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:48.285573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:48.285848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:48.296307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:48.297740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:48.314681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:48.315769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:48.316887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:48.316887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:48.316887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:48.316887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:48.316887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 634/1265] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:48.322349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:48.400993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:48.420099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13835f0c-1cb4-479d-8fe0-ff3fd9eff3c8 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:48.453070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:48.453721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:48.454004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:48.454278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:48.455001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:48.455337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:48.456707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:48.468154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 632/1266] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:48.473697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:48.571986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b5643-ac6f-4e9a-b659-ddadc198e1a1 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:48.590732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:48.605536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:48.606332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:48.606615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:48.606889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:48.614540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:48.616581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:42:48.630476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:48.631573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:48.632753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 635/1267] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:48.637641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:48.735047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e773e-5014-4160-9120-340c6d0a8b38 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:48.768567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:48.769230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:48.769513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:48.769788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:48.770623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:48.770932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:48.772911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:48.781252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:48.793073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 633/1268] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:48.800704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:48.903678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787cccde-af7d-4f26-9233-4c876ce6d2d3 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:48.936941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:48.937722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:48.938009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:48.938288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:48.948738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:48.950179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:48.967369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:48.968417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:48.969384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:48.969384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:48.969384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:48.969384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:48.969384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 636/1269] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:48.974885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:49.071633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb03e6-5af6-4aac-ab60-1172b3497f98 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:49.105885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:49.106551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:49.106833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:49.107143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:49.107872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:49.108180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:49.109554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:49.121411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 634/1270] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:49.126952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:49.158703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:49.227981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180aabf-974d-4ab3-b1c6-5aceef6b7a24 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:49.260893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:49.261670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:49.261954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:49.262230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:49.268733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:49.270174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:42:49.284779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:49.286004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:49.287535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 637/1271] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:49.292616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:49.349818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:49.390958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb40bfd8-8c75-4720-9e8a-23fbc6862486 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:49.423240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:49.423836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:49.424120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:49.424395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:49.425179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:49.425492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:49.426916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f7729-fc85-4b41-9146-0fbcc5ea1ad5 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:49.439042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 635/1272] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:49.444983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:49.533362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:49.542769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa360471-ba63-46be-9e38-3182e34ed996 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:49.575331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b43ffd-d922-4304-8ec8-6ffd27ed0b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:49.576204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43ffd-d922-4304-8ec8-6ffd27ed0b1b None admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:49.576483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43ffd-d922-4304-8ec8-6ffd27ed0b1b None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:49.576756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b43ffd-d922-4304-8ec8-6ffd27ed0b1b None admin] PATH_INFO: `/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/06938dbfe3df4bc28f065c82315e4308/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:49.578688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43ffd-d922-4304-8ec8-6ffd27ed0b1b None admin] RBAC: Authorizing `identity:create_grant(project_id=d00662c0e4ca4709b42718633ad60b71, user_id=06938dbfe3df4bc28f065c82315e4308, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:49.580154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b43ffd-d922-4304-8ec8-6ffd27ed0b1b None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:49.592981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 638/1273] 10.222.0.86 () {58 vars in 1439 bytes} [Wed Jul 20 21:42:49 2022] PUT /identity/v3/projects/d00662c0e4ca4709b42718633ad60b71/users/06938dbfe3df4bc28f065c82315e4308/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:42:51.632508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dee1e0-4de3-4b21-bd77-169b13870098 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:51.633063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dee1e0-4de3-4b21-bd77-169b13870098 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:51.633063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dee1e0-4de3-4b21-bd77-169b13870098 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:51.634416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 636/1274] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:42:51.639534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:51.639967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:51.640082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:51.656887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:51.658065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:51.723636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 639/1275] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:51.748665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dee1e0-4de3-4b21-bd77-169b13870098 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:51.847291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dee1e0-4de3-4b21-bd77-169b13870098 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:51.864526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:51.881387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:51.882169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:51.882453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:51.882777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:51.892599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 21:42:51.900792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:51.901982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:51.903052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 637/1276] 10.222.0.86 () {56 vars in 1096 bytes} [Wed Jul 20 21:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:51.907920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:51.986317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:52.004718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be2d30-05e3-4dfa-b55b-1701c75a3a64 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:52.038304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:52.038960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:52.039275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:52.039551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:52.040295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:52.040601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:52.041960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:52.049777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 640/1277] 10.222.0.86 () {56 vars in 1099 bytes} [Wed Jul 20 21:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:52.055696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:52.154007 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c3034-d45f-4a2f-b5ac-c55ea72a1bc4 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:52.186696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:52.187507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:52.187791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:52.188066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:52.199671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:52.201129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:52.218233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:52.219402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:52.220529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:52.220529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:52.220529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:52.220529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:52.220529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 638/1278] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:52.225833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:52.324230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef747-87ad-42d5-accd-6d04094abd06 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:52.357810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:52.358463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:52.358748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:52.359026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:52.359782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:52.360093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:52.361448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:52.373936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 641/1279] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:52.379546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:52.409235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:52.476101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9051e58-a27c-40db-a6fb-690d8c31ad13 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:52.508673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:52.509454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:52.509734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:52.510006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:52.517290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:52.518730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 21:42:52.532523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:52.533657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:52.534674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 639/1280] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:52.539457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:52.543155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:52.637280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600466b3-febd-4129-b7d8-2696b21a861e None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:52.670661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:52.671292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:52.671575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:52.671848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:52.672680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:52.672991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:52.674397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:52.691578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 642/1281] 10.222.0.86 () {56 vars in 1143 bytes} [Wed Jul 20 21:42:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:52.698008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:52.785398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:52.795201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82d5f-7614-4137-815f-4a5be00c190a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:52.828486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:52.829116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:52.829397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:52.829668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:52.894905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:52.896415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:52.980468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:52.991665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 640/1282] 10.222.0.86 () {58 vars in 1282 bytes} [Wed Jul 20 21:42:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 21:42:52.997488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:53.057008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:53.094006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c4fb3-f369-4cce-b563-fb14ccd2f493 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:53.129158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:53.129944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:53.130225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:53.130547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:53.140631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:53.142067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 21:42:53.159030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:53.159942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:53.160970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 21:42:53.160970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 21:42:53.160970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 21:42:53.160970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:53.160970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 643/1283] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:53.166517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:53.261834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd7e67-5d0f-464f-abcf-dcd63d707811 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:53.295079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:53.295730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:53.296008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:53.296279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:53.296990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:53.297292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:53.298648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:53.309467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 641/1284] 10.222.0.86 () {56 vars in 1109 bytes} [Wed Jul 20 21:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:53.315029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:53.414538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdc378-b15b-4981-9575-ca13c89da772 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:53.428384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:53.448494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:53.449274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:53.449558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:53.449834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:53.456318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:53.457740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 21:42:53.472162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:53.473212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: ERROR keystone.server.flask.application Jul 20 21:42:53.474180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 644/1285] 10.222.0.86 () {56 vars in 1111 bytes} [Wed Jul 20 21:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:53.479665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:53.528481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:53.577527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada99799-2b20-41e4-8e56-3f4b3a2d48ba None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:53.609111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:53.609751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:53.610028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:53.610298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:53.611100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:53.611405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:53.612820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9113ba32-8358-4345-8420-a8a1b022de2c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:53.625118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 642/1286] 10.222.0.86 () {56 vars in 1148 bytes} [Wed Jul 20 21:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:53.632185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:53.737912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8555e98-caa0-4053-b351-115f92d59fce None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:53.772607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:53.773247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:53.773524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:53.773793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:53.774610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] RBAC: Adding query filter params (role.id=62c943ce95d54a1999a447dca6a1103b, scope.project.id=d00662c0e4ca4709b42718633ad60b71, user.id=06938dbfe3df4bc28f065c82315e4308) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:53.774917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:53.776351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d76a967-3339-45b3-a1c1-44941d85ddfb None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:53.782042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 645/1287] 10.222.0.86 () {56 vars in 1374 bytes} [Wed Jul 20 21:42:53 2022] GET /identity/v3/role_assignments?user.id=06938dbfe3df4bc28f065c82315e4308&scope.project.id=d00662c0e4ca4709b42718633ad60b71&role.id=62c943ce95d54a1999a447dca6a1103b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:55.819475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3ce9-18a6-48c7-aab2-1e4e54d3b5b4 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:55.820043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3ce9-18a6-48c7-aab2-1e4e54d3b5b4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:55.820043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce3ce9-18a6-48c7-aab2-1e4e54d3b5b4 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:55.821393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 643/1288] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:42:55.826879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:55.827429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:55.827429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:55.844342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:55.845527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:55.875016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:55.912468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 646/1289] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 21:42:55.935277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce3ce9-18a6-48c7-aab2-1e4e54d3b5b4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:56.031165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce3ce9-18a6-48c7-aab2-1e4e54d3b5b4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:56.041714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:56.064718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:56.065499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:56.065782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:56.066049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] PATH_INFO: `/v3/services/placement` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:56.066688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:56.068107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:42:56.079924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:42:56.080943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:42:56.081848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 644/1290] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:42:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:42:56.088999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:56.189687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa1763-a7a6-49c6-afe8-64036c86a5cc None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:56.222955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:56.223633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:56.223920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:56.224784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:56.225495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:56.225805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:56.227195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff1a63-9cc6-4324-9f30-2e033b6f27ef None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:56.231956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 647/1291] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Jul 20 21:42:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:56.237626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:56.335846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24c6b5-1ec3-4ad3-a4f8-92d493910d6a None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:56.369676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:56.370331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:56.370616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:56.370892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] PATH_INFO: `/v3/services` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:56.371628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:42:56.371937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:56.373309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600a2ba-22cd-42f0-9ffe-45297f1e03a9 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:56.377679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 645/1292] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Jul 20 21:42:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:42:58.384359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc65e-3379-4be7-992b-8b9230b3565b None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:58.384886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc65e-3379-4be7-992b-8b9230b3565b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:58.384886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bc65e-3379-4be7-992b-8b9230b3565b None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:58.386187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 648/1293] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:42:58.391475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b43db-5303-46a9-a02e-d7d3e6bc643b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:58.391899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b43db-5303-46a9-a02e-d7d3e6bc643b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:58.392035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b43db-5303-46a9-a02e-d7d3e6bc643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:58.408984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8d2b43db-5303-46a9-a02e-d7d3e6bc643b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:42:58.410168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8d2b43db-5303-46a9-a02e-d7d3e6bc643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:42:58.438798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:58.475841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 646/1294] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:42:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:42:58.498501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc65e-3379-4be7-992b-8b9230b3565b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:42:58.529620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:42:58.598582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bc65e-3379-4be7-992b-8b9230b3565b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:42:58.632381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74d4ea-94c5-4dfe-a098-057895bdc3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:42:58.633027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74d4ea-94c5-4dfe-a098-057895bdc3a4 None admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:42:58.633306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74d4ea-94c5-4dfe-a098-057895bdc3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:42:58.633576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74d4ea-94c5-4dfe-a098-057895bdc3a4 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:42:58.634294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74d4ea-94c5-4dfe-a098-057895bdc3a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:42:58.635728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74d4ea-94c5-4dfe-a098-057895bdc3a4 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:42:58.647899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 649/1295] 10.222.0.86 () {60 vars in 1141 bytes} [Wed Jul 20 21:42:58 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:43:00.687505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5dc6d-fdff-4f9d-9fdf-bc54d7d6329c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:00.688038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5dc6d-fdff-4f9d-9fdf-bc54d7d6329c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:00.688160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5dc6d-fdff-4f9d-9fdf-bc54d7d6329c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:00.689445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 647/1296] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:43:00.695260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:00.695674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:00.695811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:00.712594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:43:00.713769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:43:00.785504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 650/1297] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:43:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:43:00.814972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5dc6d-fdff-4f9d-9fdf-bc54d7d6329c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:00.921763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5dc6d-fdff-4f9d-9fdf-bc54d7d6329c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:00.955411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:00.956142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:00.956428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:00.956702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:00.957353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:00.958758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:43:00.970355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:43:00.971532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:43:00.972497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 648/1298] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:43:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:43:00.977754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:00.996975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:43:01.077268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3be1d-acbe-4a1f-aa88-9d78e61d2f40 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:01.111593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:01.112251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:01.112537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:01.112813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:01.113785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:43:01.114095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:01.116138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:01.121343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 651/1299] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Jul 20 21:43:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:43:01.127243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:01.163083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:43:01.236497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232896c-cac2-4c13-a080-d1c17f925e00 None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:01.270209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:01.270856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:01.271634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:01.271918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:01.272711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] RBAC: Adding query filter params (interface=public, service_id=2dde9c043143417ca3e8507fabbaafaf, region_id=RegionOne) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:43:01.273026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:01.274404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a950c3-ed5e-4f6a-b50e-4fc74250181e None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:01.279321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 649/1300] 10.222.0.86 () {56 vars in 1251 bytes} [Wed Jul 20 21:43:01 2022] GET /identity/v3/endpoints?service_id=2dde9c043143417ca3e8507fabbaafaf&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:43:01.284986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:01.365613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:43:01.383309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b093b62-32cb-48c1-8d8b-6e0fe8c52221 None admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:01.416526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c6387-1b36-4c4c-b931-13e00fe001ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:01.417176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c6387-1b36-4c4c-b931-13e00fe001ef None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:01.417460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c6387-1b36-4c4c-b931-13e00fe001ef None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:01.417737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c6387-1b36-4c4c-b931-13e00fe001ef None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:01.418413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c6387-1b36-4c4c-b931-13e00fe001ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:01.419797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c6387-1b36-4c4c-b931-13e00fe001ef None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:01.426476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 652/1301] 10.222.0.86 () {56 vars in 1087 bytes} [Wed Jul 20 21:43:01 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:43:03.441022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f3e69-f4f2-452e-806e-1d3900bc6f37 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:03.441541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f3e69-f4f2-452e-806e-1d3900bc6f37 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:03.441541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f3e69-f4f2-452e-806e-1d3900bc6f37 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:03.443447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 650/1302] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:43:03.449306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:03.449687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:03.449859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:03.467470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:43:03.468651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:43:03.534862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 653/1303] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:43:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 21:43:03.558165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f3e69-f4f2-452e-806e-1d3900bc6f37 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:03.651132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:43:03.657916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f3e69-f4f2-452e-806e-1d3900bc6f37 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:03.689795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:03.690579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:03.690857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:03.691203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] PATH_INFO: `/v3/services/placement` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:03.691800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:03.693809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 21:43:03.705368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 21:43:03.706538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application with Lock( Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self._enter() Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application return self.creator() Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application created_value = creator( Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: ERROR keystone.server.flask.application Jul 20 21:43:03.707653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 651/1304] 10.222.0.86 () {56 vars in 1117 bytes} [Wed Jul 20 21:43:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 21:43:03.713349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:03.789226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:43:03.816913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec7ea3-73e7-416c-8cb6-338227263020 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:03.854758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:03.855486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:03.855835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:03.856026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] PATH_INFO: `/v3/services` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:03.856727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] RBAC: Adding query filter params (name=placement) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:43:03.857039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:03.858407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44694db1-eb69-44d0-a224-1c6b31dec67e None admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:03.864026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 654/1305] 10.222.0.86 () {56 vars in 1116 bytes} [Wed Jul 20 21:43:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:43:03.870410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:43:03.967516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bef91-1b64-49bb-89e4-41c6db11661c None admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:43:04.000109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1f46c-3b28-47fd-8df8-c7a0d552d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:43:04.000772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1f46c-3b28-47fd-8df8-c7a0d552d80f None admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:43:04.001055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1f46c-3b28-47fd-8df8-c7a0d552d80f None admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:43:04.001332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1f46c-3b28-47fd-8df8-c7a0d552d80f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:43:04.002080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1f46c-3b28-47fd-8df8-c7a0d552d80f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:43:04.003482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1f46c-3b28-47fd-8df8-c7a0d552d80f None admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:43:04.021407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 652/1306] 10.222.0.86 () {60 vars in 1144 bytes} [Wed Jul 20 21:43:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:43:13.861059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=30 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:43:14.017094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:06.807199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08daa-29a2-4e1c-bd76-5a05071be726 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:06.807747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08daa-29a2-4e1c-bd76-5a05071be726 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:06.807747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08daa-29a2-4e1c-bd76-5a05071be726 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:06.814170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e21d53-d37d-4baf-a3ea-5cb9604498be None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:06.814630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e21d53-d37d-4baf-a3ea-5cb9604498be None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:06.814731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e21d53-d37d-4baf-a3ea-5cb9604498be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:06.818930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 655/1307] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:06.889680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a1e21d53-d37d-4baf-a3ea-5cb9604498be None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:06.890896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a1e21d53-d37d-4baf-a3ea-5cb9604498be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:07.108059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 653/1308] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:44:07.147240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ea46-b013-4653-8539-a1649a6ea727 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:07.147528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ea46-b013-4653-8539-a1649a6ea727 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:07.147808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857ea46-b013-4653-8539-a1649a6ea727 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:07.150035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 656/1309] 10.222.0.86 () {54 vars in 965 bytes} [Wed Jul 20 21:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:07.157066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60693a6-5758-4954-a4e4-6030c91bc4b9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:07.157411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60693a6-5758-4954-a4e4-6030c91bc4b9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:07.157626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60693a6-5758-4954-a4e4-6030c91bc4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:07.250696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a60693a6-5758-4954-a4e4-6030c91bc4b9 None None] MFA Rules not processed for user `7f38c9f8b5ea438dbae61a15c1e84e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:07.252030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a60693a6-5758-4954-a4e4-6030c91bc4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:07.274361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:07.336137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 654/1310] 10.222.0.86 () {58 vars in 1062 bytes} [Wed Jul 20 21:44:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:07.345107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ea46-b013-4653-8539-a1649a6ea727 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:07.346892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f857ea46-b013-4653-8539-a1649a6ea727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:07.463383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857ea46-b013-4653-8539-a1649a6ea727 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:07.543048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:07.543842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:07.544131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:07.544409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:07.546082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:07.604938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:07.667264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:07.668712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271860c-b0ee-498b-a3d0-fd36142fe2cb service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:07.691760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 657/1311] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:10.049935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6a641-77ff-4f48-b6f8-dcbdeab96e91 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:10.050479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6a641-77ff-4f48-b6f8-dcbdeab96e91 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:10.050479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd6a641-77ff-4f48-b6f8-dcbdeab96e91 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:10.052719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 655/1312] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:10.058010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45640202-8a47-4d6b-9b2e-8fb543f9613a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:10.058449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45640202-8a47-4d6b-9b2e-8fb543f9613a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:10.058631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45640202-8a47-4d6b-9b2e-8fb543f9613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:10.147929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-45640202-8a47-4d6b-9b2e-8fb543f9613a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:10.149166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-45640202-8a47-4d6b-9b2e-8fb543f9613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:10.257132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 658/1313] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:10.312129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fea3f-2052-49d2-bfea-fcbff9713c9d None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:10.312325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fea3f-2052-49d2-bfea-fcbff9713c9d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:10.312636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fea3f-2052-49d2-bfea-fcbff9713c9d None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:10.314034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 656/1314] 10.222.0.86 () {54 vars in 965 bytes} [Wed Jul 20 21:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:10.320233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a4aea-5b98-49c1-9239-da0897f6167c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:10.320525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a4aea-5b98-49c1-9239-da0897f6167c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:10.320804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a4aea-5b98-49c1-9239-da0897f6167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:10.331527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:10.401811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-747a4aea-5b98-49c1-9239-da0897f6167c None None] MFA Rules not processed for user `7f38c9f8b5ea438dbae61a15c1e84e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:10.403071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-747a4aea-5b98-49c1-9239-da0897f6167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:10.484412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 659/1315] 10.222.0.86 () {58 vars in 1062 bytes} [Wed Jul 20 21:44:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:44:10.497852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fea3f-2052-49d2-bfea-fcbff9713c9d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:10.499421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-db7fea3f-2052-49d2-bfea-fcbff9713c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:10.601570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fea3f-2052-49d2-bfea-fcbff9713c9d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:10.670458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:10.673384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:10.674208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:10.674545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:10.675004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:10.676737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:10.784085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:10.786096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ce0b9-c79a-4441-aa30-3f043f8e1bf1 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:10.805549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 657/1316] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:13.372084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303d93-0753-4033-9207-fcca6a64fa1b None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:13.372615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303d93-0753-4033-9207-fcca6a64fa1b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:13.372615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303d93-0753-4033-9207-fcca6a64fa1b None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:13.374351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 660/1317] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:13.380378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40050f24-30f3-44de-ae6d-cfa244d0a224 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:13.380812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40050f24-30f3-44de-ae6d-cfa244d0a224 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:13.380948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40050f24-30f3-44de-ae6d-cfa244d0a224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:13.445569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-40050f24-30f3-44de-ae6d-cfa244d0a224 None None] MFA Rules not processed for user `d75f48bf9c68429b994a473847b8ebb0`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:13.446759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-40050f24-30f3-44de-ae6d-cfa244d0a224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:13.497523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:13.542137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 658/1318] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:13.581953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303d93-0753-4033-9207-fcca6a64fa1b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:13.584110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1b303d93-0753-4033-9207-fcca6a64fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:13.685617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303d93-0753-4033-9207-fcca6a64fa1b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:13.688486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:13.761470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:13.762124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:13.762402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:13.762672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:13.764374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:13.878534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:13.880676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14081a1-5efa-4205-bf02-5773e858e69c service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:13.899843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 661/1319] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:16.640664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82594-be1b-4a14-9b9e-bf9280c37126 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:16.641228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82594-be1b-4a14-9b9e-bf9280c37126 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:16.641228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd82594-be1b-4a14-9b9e-bf9280c37126 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:16.642586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 659/1320] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:16.647826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e47cd-84c0-461a-a28f-b3161f6a3b7d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:16.648240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e47cd-84c0-461a-a28f-b3161f6a3b7d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:16.648387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e47cd-84c0-461a-a28f-b3161f6a3b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:16.687524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:16.713028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-744e47cd-84c0-461a-a28f-b3161f6a3b7d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:16.714250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-744e47cd-84c0-461a-a28f-b3161f6a3b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:16.811516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 662/1321] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:16.850955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82594-be1b-4a14-9b9e-bf9280c37126 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:16.852532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-efd82594-be1b-4a14-9b9e-bf9280c37126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:16.956177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd82594-be1b-4a14-9b9e-bf9280c37126 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:17.027999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:17.028641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:17.028926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:17.029203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:17.030847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:17.038619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:17.139356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:17.140809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ec6d3-3dd3-44b5-84f1-a23cc9ab6270 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:17.158460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 660/1322] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:19.905497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb53490-e057-404b-9130-c41c0491ef81 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:19.906070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb53490-e057-404b-9130-c41c0491ef81 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:19.906070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb53490-e057-404b-9130-c41c0491ef81 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:19.907464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 663/1323] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:19.912852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a37c-9e73-4f16-9073-3d7fb74ef50f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:19.913253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a37c-9e73-4f16-9073-3d7fb74ef50f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:19.913434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02a37c-9e73-4f16-9073-3d7fb74ef50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:19.979326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5a02a37c-9e73-4f16-9073-3d7fb74ef50f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:19.980558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5a02a37c-9e73-4f16-9073-3d7fb74ef50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:20.039839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:20.084857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 661/1324] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:44:20.123479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb53490-e057-404b-9130-c41c0491ef81 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:20.125048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0eb53490-e057-404b-9130-c41c0491ef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:20.239808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb53490-e057-404b-9130-c41c0491ef81 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:20.252564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:20.320893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:20.321633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:20.321911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:20.322181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:20.324146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:20.440814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:20.442166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786262b-ac5b-493b-a72f-ec49d7315532 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:20.460928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 664/1325] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:22.586593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdc1a0-2f90-4dd8-8b1f-8f1ccedf52a7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:22.587186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdc1a0-2f90-4dd8-8b1f-8f1ccedf52a7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:22.587186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffdc1a0-2f90-4dd8-8b1f-8f1ccedf52a7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:22.588525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 662/1326] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:22.597262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f23e41-bab9-4fd0-a626-ea33b58bfe84 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:22.598389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f23e41-bab9-4fd0-a626-ea33b58bfe84 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:22.598555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f23e41-bab9-4fd0-a626-ea33b58bfe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:22.648211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:22.667650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b9f23e41-bab9-4fd0-a626-ea33b58bfe84 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:22.668881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b9f23e41-bab9-4fd0-a626-ea33b58bfe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:22.767836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 665/1327] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:22.805306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdc1a0-2f90-4dd8-8b1f-8f1ccedf52a7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:22.806920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9ffdc1a0-2f90-4dd8-8b1f-8f1ccedf52a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:22.911521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffdc1a0-2f90-4dd8-8b1f-8f1ccedf52a7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:22.991761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:22.992425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:22.992708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:22.992984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:22.994650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:23.013824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:23.110858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:23.112387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c37f1-4b24-4286-abbe-b7c8ca142418 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:23.138700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 663/1328] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:25.236335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7beb-5019-4411-b6bf-8d7dcfd59984 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:25.236879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7beb-5019-4411-b6bf-8d7dcfd59984 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:25.236980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b7beb-5019-4411-b6bf-8d7dcfd59984 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:25.238467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 666/1329] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:25.244376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebe472-0cf6-46c2-8ada-f908697b944b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:25.244904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebe472-0cf6-46c2-8ada-f908697b944b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:25.245088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebe472-0cf6-46c2-8ada-f908697b944b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:25.311650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-71ebe472-0cf6-46c2-8ada-f908697b944b None None] MFA Rules not processed for user `d75f48bf9c68429b994a473847b8ebb0`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:25.312843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-71ebe472-0cf6-46c2-8ada-f908697b944b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:25.382137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:25.414563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 664/1330] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:25.452448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7beb-5019-4411-b6bf-8d7dcfd59984 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:25.453998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-198b7beb-5019-4411-b6bf-8d7dcfd59984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:25.561169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b7beb-5019-4411-b6bf-8d7dcfd59984 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:25.583046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:25.636634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:25.637282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:25.637558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:25.637846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:25.639507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:25.758517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:25.760023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf118ae-b6d9-42ae-8bca-e20836e9d770 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:25.778445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 667/1331] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:28.087286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad93f3-4389-4f0f-a096-e19f4070f591 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:28.087739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad93f3-4389-4f0f-a096-e19f4070f591 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:28.087835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad93f3-4389-4f0f-a096-e19f4070f591 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:28.089218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 665/1332] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:28.094602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab23d2d-b074-494e-8141-b0be987a5ede None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:28.095090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab23d2d-b074-494e-8141-b0be987a5ede None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:28.095190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab23d2d-b074-494e-8141-b0be987a5ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:28.157952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:28.161382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-bab23d2d-b074-494e-8141-b0be987a5ede None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:28.163212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bab23d2d-b074-494e-8141-b0be987a5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:28.261400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 668/1333] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:28.298373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad93f3-4389-4f0f-a096-e19f4070f591 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:28.299974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e7ad93f3-4389-4f0f-a096-e19f4070f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:28.405428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad93f3-4389-4f0f-a096-e19f4070f591 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:28.479356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:28.480004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:28.480284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:28.480556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:28.482193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:28.510681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:28.603549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:28.605009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8bbfce-f922-4f78-9c56-183dc2a14826 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:28.624688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 666/1334] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:30.725191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935469b-23f5-45b7-8aa1-dde48bff1efd None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:30.725725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935469b-23f5-45b7-8aa1-dde48bff1efd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:30.725725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935469b-23f5-45b7-8aa1-dde48bff1efd None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:30.727141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 669/1335] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:30.732229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcd30e-1797-499b-8f1b-4d09e6fd5be7 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:30.732634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcd30e-1797-499b-8f1b-4d09e6fd5be7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:30.732779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabcd30e-1797-499b-8f1b-4d09e6fd5be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:30.796354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-cabcd30e-1797-499b-8f1b-4d09e6fd5be7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:30.797477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cabcd30e-1797-499b-8f1b-4d09e6fd5be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:30.871574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:30.894586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 667/1336] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:30.935199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935469b-23f5-45b7-8aa1-dde48bff1efd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:30.937072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8935469b-23f5-45b7-8aa1-dde48bff1efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:31.039812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935469b-23f5-45b7-8aa1-dde48bff1efd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:31.068623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:31.113937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:31.114579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:31.114862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:31.115421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:31.117085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:31.225094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:31.226554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89956b01-b068-4eeb-adaf-d62799412c9d service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:31.245291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 670/1337] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:33.592583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6eebc-bfbf-4841-8848-5673c9d53a5a None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:33.593154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6eebc-bfbf-4841-8848-5673c9d53a5a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:33.593154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6eebc-bfbf-4841-8848-5673c9d53a5a None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:33.594537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 668/1338] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:33.599864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5de9-4b05-4c3d-b84e-34535901de02 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:33.600284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5de9-4b05-4c3d-b84e-34535901de02 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:33.600404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f5de9-4b05-4c3d-b84e-34535901de02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:33.663405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ce5f5de9-4b05-4c3d-b84e-34535901de02 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:33.664608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ce5f5de9-4b05-4c3d-b84e-34535901de02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:33.672718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:33.763212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 671/1339] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:33.797711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6eebc-bfbf-4841-8848-5673c9d53a5a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:33.799509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1be6eebc-bfbf-4841-8848-5673c9d53a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:33.905631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6eebc-bfbf-4841-8848-5673c9d53a5a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:33.978334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:33.978978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:33.979355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:33.979634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:33.981293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:34.015804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:34.089039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:34.090490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03561f97-8418-4e4f-975f-ec2f3ac6a13c service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:34.109031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 669/1340] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:44:38.087215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8baed-08d6-4e79-ba4b-05fe4caf610d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:38.087674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8baed-08d6-4e79-ba4b-05fe4caf610d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:38.087774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8baed-08d6-4e79-ba4b-05fe4caf610d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:38.089335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 672/1341] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:38.094568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc3ae-bd2e-44e1-910c-48eed387e85f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:38.095015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc3ae-bd2e-44e1-910c-48eed387e85f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:38.095228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fc3ae-bd2e-44e1-910c-48eed387e85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:38.162445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-069fc3ae-bd2e-44e1-910c-48eed387e85f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:38.163607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-069fc3ae-bd2e-44e1-910c-48eed387e85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:38.249069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:38.261344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 670/1342] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:38.301579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8baed-08d6-4e79-ba4b-05fe4caf610d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:38.303157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9ce8baed-08d6-4e79-ba4b-05fe4caf610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:38.406717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8baed-08d6-4e79-ba4b-05fe4caf610d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:38.444035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:38.480027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:38.480672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:38.480950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:38.481219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:38.482852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:38.594252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:38.595752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542d3d2-039b-4f54-9a04-88d3d0484007 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:38.616805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 673/1343] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:41.336069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77cea9-f700-457a-b249-8d7b22ef5d49 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:41.336609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77cea9-f700-457a-b249-8d7b22ef5d49 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:41.336609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77cea9-f700-457a-b249-8d7b22ef5d49 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:41.337970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 671/1344] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:41.343854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0b8ec-4507-4516-b06a-5ea4e3d7f5bc None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:41.344393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0b8ec-4507-4516-b06a-5ea4e3d7f5bc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:41.344393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0b8ec-4507-4516-b06a-5ea4e3d7f5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:41.412673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d9b0b8ec-4507-4516-b06a-5ea4e3d7f5bc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:41.413929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d9b0b8ec-4507-4516-b06a-5ea4e3d7f5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:41.432518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:41.514346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 674/1345] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:41.552507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77cea9-f700-457a-b249-8d7b22ef5d49 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:41.554054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3b77cea9-f700-457a-b249-8d7b22ef5d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:41.659147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77cea9-f700-457a-b249-8d7b22ef5d49 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:41.732987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:41.733642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:41.733931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:41.734202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:41.735899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:41.779737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:41.849313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:41.850766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa65fa4-1d72-4a66-9550-050d62dbf3be service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:41.868927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 672/1346] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:45.893912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441636-046b-4b42-93dd-399f8f9bd866 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:45.894521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441636-046b-4b42-93dd-399f8f9bd866 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:45.894521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4441636-046b-4b42-93dd-399f8f9bd866 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:45.895997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 675/1347] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:45.901664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72f4e1-1a44-430f-949e-c3030004c8d0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:45.902183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72f4e1-1a44-430f-949e-c3030004c8d0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:45.902295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72f4e1-1a44-430f-949e-c3030004c8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:45.975626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ac72f4e1-1a44-430f-949e-c3030004c8d0 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:45.976888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ac72f4e1-1a44-430f-949e-c3030004c8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:46.109788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:46.114908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 673/1348] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:46.156688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441636-046b-4b42-93dd-399f8f9bd866 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:44:46.158329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a4441636-046b-4b42-93dd-399f8f9bd866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:46.295042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4441636-046b-4b42-93dd-399f8f9bd866 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:44:46.354598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:46.391683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:44:46.392492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:46.392780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:46.393055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:46.395408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:46.535045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:44:46.537709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f134ff4-71ba-4241-b9b2-c921c5d8cfa6 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:44:46.563111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 676/1349] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:44:51.503490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd336-0458-4050-ad9f-3b18cc1fac97 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:51.503953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd336-0458-4050-ad9f-3b18cc1fac97 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:51.504039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dd336-0458-4050-ad9f-3b18cc1fac97 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:51.505685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 674/1350] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:51.510756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7ca8-eabd-485b-9987-b2a7982e6941 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:51.511187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7ca8-eabd-485b-9987-b2a7982e6941 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:51.511372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de7ca8-eabd-485b-9987-b2a7982e6941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:51.644613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-19de7ca8-eabd-485b-9987-b2a7982e6941 None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:51.645755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-19de7ca8-eabd-485b-9987-b2a7982e6941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:51.687543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:51.772324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 677/1351] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:44:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:56.039387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee57e4-f249-4250-bd3d-2251e9458254 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:56.039957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee57e4-f249-4250-bd3d-2251e9458254 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:56.039957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee57e4-f249-4250-bd3d-2251e9458254 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:56.041477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 675/1352] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:56.046361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615d421-e48c-42d0-a7e5-470e7e89d43c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:56.046841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615d421-e48c-42d0-a7e5-470e7e89d43c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:56.046931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615d421-e48c-42d0-a7e5-470e7e89d43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:56.132369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1615d421-e48c-42d0-a7e5-470e7e89d43c None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:56.133543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1615d421-e48c-42d0-a7e5-470e7e89d43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:56.215551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:44:56.259047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 678/1353] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:44:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:44:57.431279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de95494-9b27-4939-8ae8-bd55b6fa9955 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:57.431809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de95494-9b27-4939-8ae8-bd55b6fa9955 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:57.431809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de95494-9b27-4939-8ae8-bd55b6fa9955 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:57.433184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 676/1354] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:44:57.438493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88b79-982f-4e8f-98d7-bc095bf76a26 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:44:57.438926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88b79-982f-4e8f-98d7-bc095bf76a26 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:44:57.439087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f88b79-982f-4e8f-98d7-bc095bf76a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:44:57.515124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e8f88b79-982f-4e8f-98d7-bc095bf76a26 None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:44:57.516337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e8f88b79-982f-4e8f-98d7-bc095bf76a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:44:57.625210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 679/1355] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:44:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:01.056773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d0fe2-1b67-4e7e-8b62-cf24421ae7f2 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:01.057279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d0fe2-1b67-4e7e-8b62-cf24421ae7f2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:01.057279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d0fe2-1b67-4e7e-8b62-cf24421ae7f2 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:01.059450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 677/1356] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:01.064262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1484f1-351b-4b28-9df7-8b168c996152 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:01.064841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1484f1-351b-4b28-9df7-8b168c996152 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:01.065477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1484f1-351b-4b28-9df7-8b168c996152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:01.116090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:01.161921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-af1484f1-351b-4b28-9df7-8b168c996152 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:01.163181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-af1484f1-351b-4b28-9df7-8b168c996152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:01.266951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 680/1357] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:01.322436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b82be2-9e60-4d47-816e-7c56e4fe8d09 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:01.322707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b82be2-9e60-4d47-816e-7c56e4fe8d09 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:01.323520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b82be2-9e60-4d47-816e-7c56e4fe8d09 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:01.325314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 678/1358] 10.222.0.86 () {54 vars in 962 bytes} [Wed Jul 20 21:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:01.334491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65b8ab-3ab9-4690-bb7d-acdd53ca0778 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:01.335406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65b8ab-3ab9-4690-bb7d-acdd53ca0778 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:01.335987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65b8ab-3ab9-4690-bb7d-acdd53ca0778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:01.425086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-bc65b8ab-3ab9-4690-bb7d-acdd53ca0778 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:01.426278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bc65b8ab-3ab9-4690-bb7d-acdd53ca0778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:01.477815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:01.554094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 681/1359] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Jul 20 21:45:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:45:01.561868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b82be2-9e60-4d47-816e-7c56e4fe8d09 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:01.563437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-66b82be2-9e60-4d47-816e-7c56e4fe8d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:01.716644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b82be2-9e60-4d47-816e-7c56e4fe8d09 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:01.804827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:01.848199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:01.848845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:01.849129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:01.849402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:01.851050 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:01.946643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:01.948374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff806db0-35d8-45cb-aa48-122b149ea4ee service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:01.978222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 679/1360] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:04.251970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40440b85-e965-48f5-9164-539bef2fab05 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:04.252638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40440b85-e965-48f5-9164-539bef2fab05 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:04.252811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40440b85-e965-48f5-9164-539bef2fab05 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:04.254238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 682/1361] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:04.259583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d632c5-a4c2-4211-9346-68f49c8a76f2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:04.260117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d632c5-a4c2-4211-9346-68f49c8a76f2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:04.260117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d632c5-a4c2-4211-9346-68f49c8a76f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:04.329205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d1d632c5-a4c2-4211-9346-68f49c8a76f2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:04.330381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d1d632c5-a4c2-4211-9346-68f49c8a76f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:04.370511 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:04.436964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 680/1362] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:04.483602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40440b85-e965-48f5-9164-539bef2fab05 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:04.486316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-40440b85-e965-48f5-9164-539bef2fab05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:04.675242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40440b85-e965-48f5-9164-539bef2fab05 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:04.685513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:04.834883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:04.835570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:04.835854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:04.836135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:04.837957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:04.933577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:04.935289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d17f4b-4155-4e39-b868-8d2be18eb478 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:04.954832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 683/1363] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:07.392079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c65d18-d4e5-44ac-9aa6-9cfa20be7667 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:07.392647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c65d18-d4e5-44ac-9aa6-9cfa20be7667 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:07.392781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c65d18-d4e5-44ac-9aa6-9cfa20be7667 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:07.394428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 681/1364] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:07.399963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e7fc-cd82-4da1-8ee3-538d48b9958d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:07.400430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e7fc-cd82-4da1-8ee3-538d48b9958d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:07.400534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717e7fc-cd82-4da1-8ee3-538d48b9958d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:07.418048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:07.473850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a717e7fc-cd82-4da1-8ee3-538d48b9958d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:07.475115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a717e7fc-cd82-4da1-8ee3-538d48b9958d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:07.590899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 684/1365] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:07.639986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013d94a-1e48-449b-9607-00669e1392e1 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:07.640270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013d94a-1e48-449b-9607-00669e1392e1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:07.640731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013d94a-1e48-449b-9607-00669e1392e1 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:07.642154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 682/1366] 10.222.0.86 () {54 vars in 962 bytes} [Wed Jul 20 21:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:07.653950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e0b81-87d0-4d0d-a7d5-cbf22e5c1617 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:07.654557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e0b81-87d0-4d0d-a7d5-cbf22e5c1617 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:07.655191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e0b81-87d0-4d0d-a7d5-cbf22e5c1617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:07.754440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a37e0b81-87d0-4d0d-a7d5-cbf22e5c1617 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:07.755720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a37e0b81-87d0-4d0d-a7d5-cbf22e5c1617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:07.795921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:07.898937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 685/1367] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Jul 20 21:45:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:07.913142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013d94a-1e48-449b-9607-00669e1392e1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:07.915265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b013d94a-1e48-449b-9607-00669e1392e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:08.125242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013d94a-1e48-449b-9607-00669e1392e1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:08.149086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:08.263372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:08.264167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:08.264566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:08.264885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:08.266552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:08.371726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:08.373366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb72235-7304-4715-9d44-48c0516890b2 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:08.395112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 683/1368] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:09.376833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5845b-ef96-455b-8dda-39887264c101 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:09.378069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5845b-ef96-455b-8dda-39887264c101 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:09.378155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5845b-ef96-455b-8dda-39887264c101 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:09.379823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 686/1369] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:45:09.384931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b21d8c-13b7-4d9f-bf8e-f548b5b900c2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:09.385224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b21d8c-13b7-4d9f-bf8e-f548b5b900c2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:09.385504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b21d8c-13b7-4d9f-bf8e-f548b5b900c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:09.415964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:09.452772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-76b21d8c-13b7-4d9f-bf8e-f548b5b900c2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:09.454174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-76b21d8c-13b7-4d9f-bf8e-f548b5b900c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:09.558074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 684/1370] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:09.919915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74814c8-c4eb-4a71-b7c4-6cc753c82524 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:09.920338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74814c8-c4eb-4a71-b7c4-6cc753c82524 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:09.920725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74814c8-c4eb-4a71-b7c4-6cc753c82524 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:09.922356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 687/1371] 10.222.0.86 () {54 vars in 964 bytes} [Wed Jul 20 21:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:09.928543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe754f-6808-4f6a-96b9-50c5f54904a0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:09.928543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe754f-6808-4f6a-96b9-50c5f54904a0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:09.928543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe754f-6808-4f6a-96b9-50c5f54904a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:10.007182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b0fe754f-6808-4f6a-96b9-50c5f54904a0 None None] MFA Rules not processed for user `ee043775ebb54bed8642c9f735b46c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:10.008548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b0fe754f-6808-4f6a-96b9-50c5f54904a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:10.058073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:10.093349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 685/1372] 10.222.0.86 () {58 vars in 1061 bytes} [Wed Jul 20 21:45:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:10.101524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74814c8-c4eb-4a71-b7c4-6cc753c82524 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:10.103186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a74814c8-c4eb-4a71-b7c4-6cc753c82524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:10.214762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74814c8-c4eb-4a71-b7c4-6cc753c82524 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:10.257820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:10.294642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:10.295333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:10.295616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:10.295889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:10.297614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:10.424816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:10.426833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ef654-559e-4b40-8f96-6e5121834a68 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:10.456192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 688/1373] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:10.636275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965dd3d-3fbe-4e09-9a2d-d27a80694538 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:10.636734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965dd3d-3fbe-4e09-9a2d-d27a80694538 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:10.637000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965dd3d-3fbe-4e09-9a2d-d27a80694538 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:10.638472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 686/1374] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:10.647680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564888b-9ebf-410f-9477-249ca0d50755 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:10.647680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564888b-9ebf-410f-9477-249ca0d50755 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:10.647680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564888b-9ebf-410f-9477-249ca0d50755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:10.734064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1564888b-9ebf-410f-9477-249ca0d50755 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:10.735655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1564888b-9ebf-410f-9477-249ca0d50755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:10.759784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:10.854203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 689/1375] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:10.907989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965dd3d-3fbe-4e09-9a2d-d27a80694538 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:10.909537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b965dd3d-3fbe-4e09-9a2d-d27a80694538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:11.075370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965dd3d-3fbe-4e09-9a2d-d27a80694538 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:11.137359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:11.214305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:11.214979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:11.215298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:11.215571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:11.217303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:11.324001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:11.325627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce7c11-fd8b-4c74-882b-c0f69ae6d28a service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:11.344010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 687/1376] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:13.445653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7797890-eaf8-4bb5-b2c1-115d2cc8a1e6 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:13.446159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7797890-eaf8-4bb5-b2c1-115d2cc8a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:13.446159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7797890-eaf8-4bb5-b2c1-115d2cc8a1e6 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:13.447564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 690/1377] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:13.452811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85cd53-c844-432f-bd74-03d14db3b4ec None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:13.453204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85cd53-c844-432f-bd74-03d14db3b4ec None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:13.453374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85cd53-c844-432f-bd74-03d14db3b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:13.523910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2f85cd53-c844-432f-bd74-03d14db3b4ec None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:13.525101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2f85cd53-c844-432f-bd74-03d14db3b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:13.532137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:13.635798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 688/1378] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:13.680627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7797890-eaf8-4bb5-b2c1-115d2cc8a1e6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:13.682159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d7797890-eaf8-4bb5-b2c1-115d2cc8a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:13.835986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7797890-eaf8-4bb5-b2c1-115d2cc8a1e6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:13.839753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:13.968988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:13.970203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:13.970544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:13.970891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:13.972787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:14.104085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:14.105633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4413c18-e05a-4820-8c77-0eb8d1e531b3 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:14.126625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 691/1379] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:16.223428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9bcaa-2d18-47f1-87ce-3ff241b6d9a7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:16.223957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9bcaa-2d18-47f1-87ce-3ff241b6d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:16.224183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9bcaa-2d18-47f1-87ce-3ff241b6d9a7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:16.225778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 689/1380] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:16.230949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ae73-5980-4d85-83d9-a136cc23be1d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:16.231409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ae73-5980-4d85-83d9-a136cc23be1d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:16.231602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ae73-5980-4d85-83d9-a136cc23be1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:16.242308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:16.304595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f701ae73-5980-4d85-83d9-a136cc23be1d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:16.305816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f701ae73-5980-4d85-83d9-a136cc23be1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:16.413264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 692/1381] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:16.459093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9bcaa-2d18-47f1-87ce-3ff241b6d9a7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:16.460776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-33c9bcaa-2d18-47f1-87ce-3ff241b6d9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:16.610506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:16.623577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9bcaa-2d18-47f1-87ce-3ff241b6d9a7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:16.745862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:16.746515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:16.746823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:16.747123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:16.748768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:16.853083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:16.854808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c27573-41c6-4714-a82e-fc10942586ff service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:16.868327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:16.879947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 690/1382] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:18.976878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2949d9-8b01-4ea2-b5cb-e94d5bd81e0f None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:18.977443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2949d9-8b01-4ea2-b5cb-e94d5bd81e0f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:18.977642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2949d9-8b01-4ea2-b5cb-e94d5bd81e0f None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:18.979551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 693/1383] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:18.985087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526626a-6d16-4bb9-921f-7d8ce8cc78ff None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:18.985599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526626a-6d16-4bb9-921f-7d8ce8cc78ff None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:18.985775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526626a-6d16-4bb9-921f-7d8ce8cc78ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:19.056379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d526626a-6d16-4bb9-921f-7d8ce8cc78ff None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:19.057997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d526626a-6d16-4bb9-921f-7d8ce8cc78ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:19.170652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 691/1384] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:19.223081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2949d9-8b01-4ea2-b5cb-e94d5bd81e0f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:19.224633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7a2949d9-8b01-4ea2-b5cb-e94d5bd81e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:19.312037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:19.391992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2949d9-8b01-4ea2-b5cb-e94d5bd81e0f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:19.518879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:19.519728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:19.520082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:19.520490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:19.522636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:19.586321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:19.629852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:19.631507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9443b-9287-43db-9fce-65060d3460cc service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:19.649964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 694/1385] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:21.751792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c966-1ed1-4be4-bb2a-47184ee3cd5c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:21.752493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c966-1ed1-4be4-bb2a-47184ee3cd5c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:21.752493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c966-1ed1-4be4-bb2a-47184ee3cd5c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:21.753739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 692/1386] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:21.759145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342990bb-45ee-4f46-a59b-c5b617466b65 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:21.759770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342990bb-45ee-4f46-a59b-c5b617466b65 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:21.759770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342990bb-45ee-4f46-a59b-c5b617466b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:21.832614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-342990bb-45ee-4f46-a59b-c5b617466b65 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:21.833853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-342990bb-45ee-4f46-a59b-c5b617466b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:21.962337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 695/1387] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:22.009371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665c966-1ed1-4be4-bb2a-47184ee3cd5c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:22.012086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1665c966-1ed1-4be4-bb2a-47184ee3cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:22.111842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:22.199847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665c966-1ed1-4be4-bb2a-47184ee3cd5c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:22.340121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:22.341031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:22.341395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:22.341752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:22.343950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:22.388872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:22.442281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:22.443935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a3a5f-dfcb-426a-a56e-0c2f6f91f2be service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:22.462557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 693/1388] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:24.665236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de3a1-4367-435a-9557-25b67663a5f9 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:24.665845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de3a1-4367-435a-9557-25b67663a5f9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:24.665845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238de3a1-4367-435a-9557-25b67663a5f9 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:24.667212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 696/1389] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:24.672402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff771b2-ee0e-4730-a2c2-99e541aafddc None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:24.672822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff771b2-ee0e-4730-a2c2-99e541aafddc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:24.673081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff771b2-ee0e-4730-a2c2-99e541aafddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:24.741766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fff771b2-ee0e-4730-a2c2-99e541aafddc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:24.743039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fff771b2-ee0e-4730-a2c2-99e541aafddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:24.840967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 694/1390] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:24.885291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de3a1-4367-435a-9557-25b67663a5f9 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:24.886803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-238de3a1-4367-435a-9557-25b67663a5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:24.917011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:25.034104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238de3a1-4367-435a-9557-25b67663a5f9 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:25.140331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:25.152410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:25.153062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:25.153346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:25.153622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:25.155313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:25.247713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:25.249311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e02b5-c3d7-4d5f-b366-c61899513e4c service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:25.267983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 697/1391] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:25.684221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebb6c5-4c5a-4f65-a9a2-0844e62c6a2b None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:25.684795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebb6c5-4c5a-4f65-a9a2-0844e62c6a2b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:25.684795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebb6c5-4c5a-4f65-a9a2-0844e62c6a2b None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:25.686152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 695/1392] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:25.691739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9555b-b0cd-477c-924a-9a0ccc9c26b2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:25.692216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9555b-b0cd-477c-924a-9a0ccc9c26b2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:25.692357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9555b-b0cd-477c-924a-9a0ccc9c26b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:25.768146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-01b9555b-b0cd-477c-924a-9a0ccc9c26b2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:25.770241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-01b9555b-b0cd-477c-924a-9a0ccc9c26b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:25.836318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:25.882699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 698/1393] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:25.950384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e26f6-6345-4dbc-89cc-cd37b361468e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:25.950679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e26f6-6345-4dbc-89cc-cd37b361468e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:25.950951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e26f6-6345-4dbc-89cc-cd37b361468e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:25.952508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 696/1394] 10.222.0.86 () {54 vars in 966 bytes} [Wed Jul 20 21:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:45:25.958637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df721-12cd-4e8c-b448-bae33c4ab8ab None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:25.959017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df721-12cd-4e8c-b448-bae33c4ab8ab None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:25.959332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087df721-12cd-4e8c-b448-bae33c4ab8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:26.047685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-087df721-12cd-4e8c-b448-bae33c4ab8ab None None] MFA Rules not processed for user `be5097840a23492585d297d35e2fea20`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:26.049720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-087df721-12cd-4e8c-b448-bae33c4ab8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:26.150265 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 699/1395] 10.222.0.86 () {58 vars in 1063 bytes} [Wed Jul 20 21:45:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:26.158295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e26f6-6345-4dbc-89cc-cd37b361468e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:26.159933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-498e26f6-6345-4dbc-89cc-cd37b361468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:26.173356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:26.277675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e26f6-6345-4dbc-89cc-cd37b361468e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:26.366119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:26.366938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:26.367408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:26.367685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:26.369907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:26.440963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:26.495882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:26.497353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084069f-28e5-4e4b-b6c6-08dd79da9563 service glance] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:26.518533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 697/1396] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:27.312972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb6d9d-fd6d-4086-b920-49e7006b2626 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:27.313482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb6d9d-fd6d-4086-b920-49e7006b2626 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:27.314594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb6d9d-fd6d-4086-b920-49e7006b2626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:27.392541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-22eb6d9d-fd6d-4086-b920-49e7006b2626 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:27.393769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-22eb6d9d-fd6d-4086-b920-49e7006b2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:27.402752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:27.497201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 700/1397] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:45:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:27.543980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3aa56-8e16-4f2e-843e-d1c88f397cc4 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:27.544466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3aa56-8e16-4f2e-843e-d1c88f397cc4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:27.544660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3aa56-8e16-4f2e-843e-d1c88f397cc4 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:27.546038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 698/1398] 10.222.0.86 () {54 vars in 943 bytes} [Wed Jul 20 21:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:27.553648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02682-1e44-41f9-bab2-4be81d8c506b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:27.553945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02682-1e44-41f9-bab2-4be81d8c506b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:27.554244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02682-1e44-41f9-bab2-4be81d8c506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:27.630733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-66c02682-1e44-41f9-bab2-4be81d8c506b None None] MFA Rules not processed for user `6952f7293b2f46f28941b48429dd2e73`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:27.631996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-66c02682-1e44-41f9-bab2-4be81d8c506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:27.717626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:27.749891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 701/1399] 10.222.0.86 () {58 vars in 1040 bytes} [Wed Jul 20 21:45:27 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:45:27.759855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3aa56-8e16-4f2e-843e-d1c88f397cc4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:27.761384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1cc3aa56-8e16-4f2e-843e-d1c88f397cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:27.870043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfea0b-2644-4531-a1f6-c5351e569225 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:27.870334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfea0b-2644-4531-a1f6-c5351e569225 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:27.870616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfea0b-2644-4531-a1f6-c5351e569225 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:27.872061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 702/1400] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:45:27.879028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:27.879342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:27.879623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:27.902938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3aa56-8e16-4f2e-843e-d1c88f397cc4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:27.941563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:27.946516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:27.947778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:28.013583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:28.014157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:28.014436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:28.014709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:28.016433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:28.052498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 703/1401] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:28.098213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:28.099479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:28.109285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:28.110938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045cf9-4bb2-4633-8d32-3f733ec4fb69 service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:28.120125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 699/1402] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:45:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:45:28.170028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:28.229665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e650-614e-4241-842b-ea31a5f05d0f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:28.379787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:28.380426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:28.380716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:28.381082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:28.382906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:28.449153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:28.487046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:28.489059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec656a4c-1b9b-4ce5-9f69-7b140a6ba153 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:28.518508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 704/1403] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:45:30.667632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8866302-c92c-4f92-9a91-603aca094ab8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:30.668180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8866302-c92c-4f92-9a91-603aca094ab8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:30.668180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8866302-c92c-4f92-9a91-603aca094ab8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:30.669547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 700/1404] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:30.675450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc824a71-5a3c-4ef3-bf18-52751f28c461 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:30.675928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc824a71-5a3c-4ef3-bf18-52751f28c461 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:30.676008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc824a71-5a3c-4ef3-bf18-52751f28c461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:30.756617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-dc824a71-5a3c-4ef3-bf18-52751f28c461 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:30.758339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dc824a71-5a3c-4ef3-bf18-52751f28c461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:30.956500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 705/1405] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:30.995102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8866302-c92c-4f92-9a91-603aca094ab8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:30.996633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f8866302-c92c-4f92-9a91-603aca094ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:31.082606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:31.164274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8866302-c92c-4f92-9a91-603aca094ab8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:31.308410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:31.309155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:31.309507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:31.309787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:31.311577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:31.348145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:31.420522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:31.422138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8aff5-075d-40df-bbb5-2e844ec503af service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:31.442780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 701/1406] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:33.664347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b8a4e-0363-44e8-b4cc-bd48af47cb53 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:33.664907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b8a4e-0363-44e8-b4cc-bd48af47cb53 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:33.664907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b8a4e-0363-44e8-b4cc-bd48af47cb53 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:33.666428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 706/1407] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:45:33.672636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8c2cd-6eab-447d-81e1-b49a6cecd095 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:33.674324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8c2cd-6eab-447d-81e1-b49a6cecd095 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:33.674324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8c2cd-6eab-447d-81e1-b49a6cecd095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:33.784226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fed8c2cd-6eab-447d-81e1-b49a6cecd095 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:33.785464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fed8c2cd-6eab-447d-81e1-b49a6cecd095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:33.921319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 702/1408] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:45:33.974025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b8a4e-0363-44e8-b4cc-bd48af47cb53 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:33.975667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ce7b8a4e-0363-44e8-b4cc-bd48af47cb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:34.020055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:34.181430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b8a4e-0363-44e8-b4cc-bd48af47cb53 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:34.304825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:34.311298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:34.311992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:34.312281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:34.312560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:34.314496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:34.435686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:34.437964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615066a-687b-4344-a462-6656fc413b6c service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:34.472476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 707/1409] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:43.918435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:44.463119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=47 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:51.268539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae027266-e032-4d25-8281-03d357a218a7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:51.269131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae027266-e032-4d25-8281-03d357a218a7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:51.269226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae027266-e032-4d25-8281-03d357a218a7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:51.270699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 703/1410] 10.222.0.86 () {54 vars in 918 bytes} [Wed Jul 20 21:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:51.275990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a11ac-a454-46da-b026-1618dbdf56de None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:51.276426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a11ac-a454-46da-b026-1618dbdf56de None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:51.276522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a11ac-a454-46da-b026-1618dbdf56de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:51.351396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-873a11ac-a454-46da-b026-1618dbdf56de None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:51.352546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-873a11ac-a454-46da-b026-1618dbdf56de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:51.459053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 708/1411] 10.222.0.86 () {58 vars in 1015 bytes} [Wed Jul 20 21:45:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:51.474792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a10487-104d-4c6a-90d4-cb316614ff2d None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:51.475111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a10487-104d-4c6a-90d4-cb316614ff2d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:51.475395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a10487-104d-4c6a-90d4-cb316614ff2d None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:51.476754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 704/1412] 10.222.0.86 () {54 vars in 918 bytes} [Wed Jul 20 21:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:51.481489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77cee2-e41e-413e-8c63-70dd65bf4656 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:51.481804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77cee2-e41e-413e-8c63-70dd65bf4656 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:51.482078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77cee2-e41e-413e-8c63-70dd65bf4656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:51.560072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:51.563400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2a77cee2-e41e-413e-8c63-70dd65bf4656 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:51.565214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2a77cee2-e41e-413e-8c63-70dd65bf4656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:51.686131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 709/1413] 10.222.0.86 () {58 vars in 1015 bytes} [Wed Jul 20 21:45:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:54.251486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521f823-a2b1-4c07-aee0-e1a4ed22f9c8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:54.252021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521f823-a2b1-4c07-aee0-e1a4ed22f9c8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:54.252021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521f823-a2b1-4c07-aee0-e1a4ed22f9c8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:54.253375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:54.258582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebf563-5c2a-4960-8a96-a4d4b908de6a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:54.259080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebf563-5c2a-4960-8a96-a4d4b908de6a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:54.259193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebf563-5c2a-4960-8a96-a4d4b908de6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:54.327402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-31ebf563-5c2a-4960-8a96-a4d4b908de6a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:54.329404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-31ebf563-5c2a-4960-8a96-a4d4b908de6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:54.385169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:54.429143 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:45:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:54.473515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521f823-a2b1-4c07-aee0-e1a4ed22f9c8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:54.475392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6521f823-a2b1-4c07-aee0-e1a4ed22f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:54.623379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521f823-a2b1-4c07-aee0-e1a4ed22f9c8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:54.695770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:54.736202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:54.736851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:54.737135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:54.737411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:54.739095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:54.837266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:54.838862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd55120-e87e-4b0d-83bf-67f51f08c17e service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:54.856583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 706/1416] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:55.774946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d540c-c24b-4e4a-923f-4c5bfcecccc2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:55.775566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d540c-c24b-4e4a-923f-4c5bfcecccc2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:55.775566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d540c-c24b-4e4a-923f-4c5bfcecccc2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:55.776992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:55.782134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c6d9-1776-4912-9642-dc65715c0e73 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:55.782589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c6d9-1776-4912-9642-dc65715c0e73 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:55.782696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098c6d9-1776-4912-9642-dc65715c0e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:55.860173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4098c6d9-1776-4912-9642-dc65715c0e73 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:55.861468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4098c6d9-1776-4912-9642-dc65715c0e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:55.890171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:55.963422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:56.119397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d540c-c24b-4e4a-923f-4c5bfcecccc2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:56.120931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-559d540c-c24b-4e4a-923f-4c5bfcecccc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:56.268012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d540c-c24b-4e4a-923f-4c5bfcecccc2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:56.308630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:56.383446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:56.384011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:56.384295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:56.384569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:56.386238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:56.478279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:56.479928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab397188-ffc4-404f-8a15-fce9ca739bcb service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:56.498801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 712/1419] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:57.393439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20c8e-d14e-4e19-bf68-1dd506358b93 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:57.394068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20c8e-d14e-4e19-bf68-1dd506358b93 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:57.394068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b20c8e-d14e-4e19-bf68-1dd506358b93 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:57.395322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:57.400323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f3cbe-853d-4dd1-a8ce-caea1ff9fe21 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:57.400748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f3cbe-853d-4dd1-a8ce-caea1ff9fe21 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:57.400879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f3cbe-853d-4dd1-a8ce-caea1ff9fe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:57.457876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:57.480580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d78f3cbe-853d-4dd1-a8ce-caea1ff9fe21 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:57.481833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d78f3cbe-853d-4dd1-a8ce-caea1ff9fe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:57.581716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:45:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:57.752571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20c8e-d14e-4e19-bf68-1dd506358b93 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:57.754106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-46b20c8e-d14e-4e19-bf68-1dd506358b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:57.923756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b20c8e-d14e-4e19-bf68-1dd506358b93 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:57.935931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:58.043454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56874404-f788-4606-b224-504598ab60d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:58.044072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874404-f788-4606-b224-504598ab60d0 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:58.044360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874404-f788-4606-b224-504598ab60d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:58.044641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56874404-f788-4606-b224-504598ab60d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:58.046507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-56874404-f788-4606-b224-504598ab60d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:58.141360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56874404-f788-4606-b224-504598ab60d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:58.143085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56874404-f788-4606-b224-504598ab60d0 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:58.162463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 709/1422] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:45:59.067510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7071-76cc-4a85-9209-c771bb93f092 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:59.068097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7071-76cc-4a85-9209-c771bb93f092 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:59.068097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d7071-76cc-4a85-9209-c771bb93f092 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:59.069454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:45:59.074987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30cd6-ad83-4223-9c89-e5d80b220316 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:59.075865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30cd6-ad83-4223-9c89-e5d80b220316 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:59.075865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30cd6-ad83-4223-9c89-e5d80b220316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:59.098002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:59.156792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-51e30cd6-ad83-4223-9c89-e5d80b220316 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:45:59.158044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-51e30cd6-ad83-4223-9c89-e5d80b220316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:59.257015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:45:59.409377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d7071-76cc-4a85-9209-c771bb93f092 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:45:59.410856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f00d7071-76cc-4a85-9209-c771bb93f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:59.527760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:59.567998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d7071-76cc-4a85-9209-c771bb93f092 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:45:59.697146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:45:59.697704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:45:59.697980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:45:59.698415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:45:59.700106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:45:59.779517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:45:59.794258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:45:59.795887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd6772-bc54-48e2-8fcd-4b9fa035d39c service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:45:59.816272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 715/1425] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:00.729452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b77ae-19cf-4617-9ee2-facf9feb95a1 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:00.730023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b77ae-19cf-4617-9ee2-facf9feb95a1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:00.730023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b77ae-19cf-4617-9ee2-facf9feb95a1 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:00.731482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:00.736909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26a38-52e0-4212-82f3-2cd630c3ca20 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:00.737395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26a38-52e0-4212-82f3-2cd630c3ca20 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:00.737503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26a38-52e0-4212-82f3-2cd630c3ca20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:00.821769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f7f26a38-52e0-4212-82f3-2cd630c3ca20 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:00.822904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f7f26a38-52e0-4212-82f3-2cd630c3ca20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:00.933960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:46:01.085604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b77ae-19cf-4617-9ee2-facf9feb95a1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:01.087201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-492b77ae-19cf-4617-9ee2-facf9feb95a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:01.173936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:01.239952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b77ae-19cf-4617-9ee2-facf9feb95a1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:01.351415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:01.351985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:01.352275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:01.352549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:01.354211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:01.403183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:01.446287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:01.447960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b201c-e957-4c61-8e50-1ec330a3e644 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:01.467548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 712/1428] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:02.308420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e146-6bb8-4a31-8d26-805edd0203cd None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:02.309023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e146-6bb8-4a31-8d26-805edd0203cd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:02.309023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6e146-6bb8-4a31-8d26-805edd0203cd None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:02.310370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:02.316796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a888487-59de-450a-8938-76744828a8f3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:02.317662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a888487-59de-450a-8938-76744828a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:02.317662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a888487-59de-450a-8938-76744828a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:02.397205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4a888487-59de-450a-8938-76744828a8f3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:02.398340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4a888487-59de-450a-8938-76744828a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:02.503362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:02.657275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6e146-6bb8-4a31-8d26-805edd0203cd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:02.658737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c8d6e146-6bb8-4a31-8d26-805edd0203cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:02.708694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:02.809146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6e146-6bb8-4a31-8d26-805edd0203cd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:02.930965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:02.931566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:02.931841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:02.932107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:02.933804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:02.941940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:03.031875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:03.033555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115297d2-2251-4bee-92a6-a915d9f66144 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:03.052417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 718/1431] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:03.879995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69daa-b297-433b-b0a3-ae4f20506c0e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:03.880615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69daa-b297-433b-b0a3-ae4f20506c0e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:03.880615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d69daa-b297-433b-b0a3-ae4f20506c0e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:03.881904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:03.887199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f2f4b-f343-4189-9df9-ef8f3b4b309c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:03.887646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f2f4b-f343-4189-9df9-ef8f3b4b309c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:03.887755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f2f4b-f343-4189-9df9-ef8f3b4b309c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:03.965164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-661f2f4b-f343-4189-9df9-ef8f3b4b309c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:03.966421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-661f2f4b-f343-4189-9df9-ef8f3b4b309c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:04.038021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:04.067163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:04.225922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d69daa-b297-433b-b0a3-ae4f20506c0e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:04.227501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-74d69daa-b297-433b-b0a3-ae4f20506c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:04.245821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:04.385415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d69daa-b297-433b-b0a3-ae4f20506c0e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:04.476026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:04.502739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:04.503420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:04.503701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:04.503970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:04.505619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:04.597021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:04.598673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c770a-f60e-4881-a269-86363ffa043e service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:04.616864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 715/1434] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:05.438218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9119f21-273a-4c81-bc8d-cc01c20d926d None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:05.438752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9119f21-273a-4c81-bc8d-cc01c20d926d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:05.439204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9119f21-273a-4c81-bc8d-cc01c20d926d None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:05.440605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:05.445701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dfa88-bedd-4214-a7f9-4499243eb1e1 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:05.446172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dfa88-bedd-4214-a7f9-4499243eb1e1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:05.446280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dfa88-bedd-4214-a7f9-4499243eb1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:05.521535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-621dfa88-bedd-4214-a7f9-4499243eb1e1 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:05.522756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-621dfa88-bedd-4214-a7f9-4499243eb1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:05.565052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:05.624386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:05.784549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9119f21-273a-4c81-bc8d-cc01c20d926d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:05.786088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f9119f21-273a-4c81-bc8d-cc01c20d926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:05.954792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9119f21-273a-4c81-bc8d-cc01c20d926d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:06.011198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:06.075912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:06.076485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:06.076770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:06.077062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:06.079478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:06.181780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:06.183460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24ba76-12b0-4d9f-93d5-fdd2b1e8ea8f service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:06.205654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 721/1437] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:07.029355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafec2b9-c5d6-4635-9f0d-c9f86b9b402f None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:07.029928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafec2b9-c5d6-4635-9f0d-c9f86b9b402f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:07.029928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafec2b9-c5d6-4635-9f0d-c9f86b9b402f None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:07.031940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:07.037431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bafee-4a7c-4cf0-b418-123c55e3f541 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:07.037870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bafee-4a7c-4cf0-b418-123c55e3f541 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:07.038230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bafee-4a7c-4cf0-b418-123c55e3f541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:07.119314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e10bafee-4a7c-4cf0-b418-123c55e3f541 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:07.120553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e10bafee-4a7c-4cf0-b418-123c55e3f541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:07.122611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:07.223587 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:46:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:07.378780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafec2b9-c5d6-4635-9f0d-c9f86b9b402f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:07.380329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bafec2b9-c5d6-4635-9f0d-c9f86b9b402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:07.528077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafec2b9-c5d6-4635-9f0d-c9f86b9b402f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:07.555674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:07.647557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:07.648592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:07.648868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:07.649135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:07.650792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:07.745441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:07.747144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ee284-80af-49e2-8c89-7a1feedd261b service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:07.766415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 718/1440] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:09.863040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ced86-128d-498b-9861-09f7e1caa7c5 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:09.863673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ced86-128d-498b-9861-09f7e1caa7c5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:09.863673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ced86-128d-498b-9861-09f7e1caa7c5 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:09.865330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:09.871390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5fd75-ca18-454a-8513-22f64f535d9e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:09.872134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5fd75-ca18-454a-8513-22f64f535d9e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:09.872134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5fd75-ca18-454a-8513-22f64f535d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:09.919328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:09.947294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-98e5fd75-ca18-454a-8513-22f64f535d9e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:09.948532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-98e5fd75-ca18-454a-8513-22f64f535d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:10.052497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:46:10.093932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ced86-128d-498b-9861-09f7e1caa7c5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:10.096333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-074ced86-128d-498b-9861-09f7e1caa7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:10.207729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ced86-128d-498b-9861-09f7e1caa7c5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:10.245833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:10.292920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:10.293750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:10.294063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:10.294404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:10.297068 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:10.427973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:10.429452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a7c57e-6ab9-4a82-983e-3d73cbed073c service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:10.450332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 724/1443] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:12.770103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2acb-e6ef-4ea4-b9f5-1c7c3d7985d3 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:12.770646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2acb-e6ef-4ea4-b9f5-1c7c3d7985d3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:12.770646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2acb-e6ef-4ea4-b9f5-1c7c3d7985d3 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:12.772066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:12.777386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914c2b-f5ae-4aec-a96c-8fe5466269b8 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:12.777823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914c2b-f5ae-4aec-a96c-8fe5466269b8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:12.778048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f914c2b-f5ae-4aec-a96c-8fe5466269b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:12.846250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-8f914c2b-f5ae-4aec-a96c-8fe5466269b8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:12.847411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8f914c2b-f5ae-4aec-a96c-8fe5466269b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:12.860076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:12.947651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:12.986878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add2acb-e6ef-4ea4-b9f5-1c7c3d7985d3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:12.988538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8add2acb-e6ef-4ea4-b9f5-1c7c3d7985d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:13.100030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add2acb-e6ef-4ea4-b9f5-1c7c3d7985d3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:13.112946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:13.178997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:13.179985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:13.179985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:13.180228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:13.181834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:13.291233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:13.292674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0c42d6-ff52-4b97-b330-374a0e7672fe service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:13.310997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 721/1446] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:15.716963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e128-d545-4365-9d02-371006e7faa6 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:15.717541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e128-d545-4365-9d02-371006e7faa6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:15.717541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e128-d545-4365-9d02-371006e7faa6 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:15.718917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:15.725225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c97f-9b6c-4bef-8b29-b3c6c1b62d4c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:15.725691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c97f-9b6c-4bef-8b29-b3c6c1b62d4c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:15.725790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124c97f-9b6c-4bef-8b29-b3c6c1b62d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:15.775127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:15.795763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9124c97f-9b6c-4bef-8b29-b3c6c1b62d4c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:15.797004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9124c97f-9b6c-4bef-8b29-b3c6c1b62d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:15.907985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:15.947456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e128-d545-4365-9d02-371006e7faa6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:15.949092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d720e128-d545-4365-9d02-371006e7faa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:16.063937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e128-d545-4365-9d02-371006e7faa6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:16.109265 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:16.146948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:16.147635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:16.147920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:16.148195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:16.149881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:16.279750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:16.281205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50517a36-639d-4b9b-9329-9b06d68d1f27 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:16.301327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 727/1449] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:18.600656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899d41-f6b9-4925-8bbc-f949e88934b8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:18.601238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899d41-f6b9-4925-8bbc-f949e88934b8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:18.601238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86899d41-f6b9-4925-8bbc-f949e88934b8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:18.602576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:18.608317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af9848-06c3-4241-bd5c-fdd06233cf2d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:18.608853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af9848-06c3-4241-bd5c-fdd06233cf2d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:18.608968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af9848-06c3-4241-bd5c-fdd06233cf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:18.680563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e6af9848-06c3-4241-bd5c-fdd06233cf2d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:18.681810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e6af9848-06c3-4241-bd5c-fdd06233cf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:18.705038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:18.798396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:18.840482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86899d41-f6b9-4925-8bbc-f949e88934b8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:18.842053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-86899d41-f6b9-4925-8bbc-f949e88934b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:18.955879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86899d41-f6b9-4925-8bbc-f949e88934b8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:18.977609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:19.036992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:19.037653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:19.037922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:19.038193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:19.039898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:19.161642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:19.163013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4994b6c-bab1-458f-a939-b465432896ff service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:19.183151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 724/1452] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:21.401089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd04ad8-84e5-4215-a3f5-0a29d2b35f8e None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:21.401632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd04ad8-84e5-4215-a3f5-0a29d2b35f8e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:21.401632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd04ad8-84e5-4215-a3f5-0a29d2b35f8e None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:21.403173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:21.409370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df180d8-7643-48b6-8f57-7fcbb77e8368 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:21.409845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df180d8-7643-48b6-8f57-7fcbb77e8368 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:21.409943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df180d8-7643-48b6-8f57-7fcbb77e8368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:21.477552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:21.480442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7df180d8-7643-48b6-8f57-7fcbb77e8368 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:21.481764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7df180d8-7643-48b6-8f57-7fcbb77e8368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:21.592118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:21.631340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd04ad8-84e5-4215-a3f5-0a29d2b35f8e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:21.632916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fdd04ad8-84e5-4215-a3f5-0a29d2b35f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:21.765518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd04ad8-84e5-4215-a3f5-0a29d2b35f8e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:21.828276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:21.854282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:21.854929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:21.855248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:21.855521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:21.857192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:21.972255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:21.973703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9467584-80c1-4663-9c19-3b7670788e31 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:21.994489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 730/1455] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:24.247118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca86f9-ddc8-479b-b6e2-04cd4838bbf2 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:24.247691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca86f9-ddc8-479b-b6e2-04cd4838bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:24.247691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca86f9-ddc8-479b-b6e2-04cd4838bbf2 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:24.249084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:24.255502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7375-6e70-4a7d-8cda-61fd9bcd5585 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:24.255967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7375-6e70-4a7d-8cda-61fd9bcd5585 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:24.256066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c7375-6e70-4a7d-8cda-61fd9bcd5585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:24.323879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-232c7375-6e70-4a7d-8cda-61fd9bcd5585 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:24.325102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-232c7375-6e70-4a7d-8cda-61fd9bcd5585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:24.362021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:24.439703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:46:24.479485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca86f9-ddc8-479b-b6e2-04cd4838bbf2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:24.481037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a4ca86f9-ddc8-479b-b6e2-04cd4838bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:24.588017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca86f9-ddc8-479b-b6e2-04cd4838bbf2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:24.618252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:24.671422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:24.672621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:24.672953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:24.673234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:24.674914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:24.792108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:24.793972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9afd1c-9a73-410d-a536-44965c107cfa service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:24.815885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 727/1458] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:27.438176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b071-740c-4c3f-acc9-aa095a9fd485 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:27.438887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b071-740c-4c3f-acc9-aa095a9fd485 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:27.438887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b071-740c-4c3f-acc9-aa095a9fd485 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:27.440705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:27.446360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae8c1-4181-42fc-9ae1-79f89041b716 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:27.446895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae8c1-4181-42fc-9ae1-79f89041b716 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:27.447039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ae8c1-4181-42fc-9ae1-79f89041b716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:27.515765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-997ae8c1-4181-42fc-9ae1-79f89041b716 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:27.516993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-997ae8c1-4181-42fc-9ae1-79f89041b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:27.525301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:27.621062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:27.665667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b071-740c-4c3f-acc9-aa095a9fd485 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:27.667314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5e77b071-740c-4c3f-acc9-aa095a9fd485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:27.787605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b071-740c-4c3f-acc9-aa095a9fd485 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:27.859731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:27.879637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:27.880281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:27.880563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:27.880827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:27.882640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:28.117437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:28.118902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8638b8d-c5a7-4c03-ba3e-8818dfcfb56b service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:28.137526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 733/1461] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:30.274072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13624e0-f3b9-469b-b0b2-2f4412d37248 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:30.274613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13624e0-f3b9-469b-b0b2-2f4412d37248 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:30.274613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13624e0-f3b9-469b-b0b2-2f4412d37248 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:30.276041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:30.282353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c583e3-2df2-4a9b-8abf-8a405c64c3f9 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:30.282817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c583e3-2df2-4a9b-8abf-8a405c64c3f9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:30.282925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c583e3-2df2-4a9b-8abf-8a405c64c3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:30.356000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d1c583e3-2df2-4a9b-8abf-8a405c64c3f9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:30.357138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d1c583e3-2df2-4a9b-8abf-8a405c64c3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:30.396415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:30.457230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:30.495509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13624e0-f3b9-469b-b0b2-2f4412d37248 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:30.497047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b13624e0-f3b9-469b-b0b2-2f4412d37248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:30.604201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13624e0-f3b9-469b-b0b2-2f4412d37248 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:30.645058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:30.681376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:30.682020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:30.682300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:30.682573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:30.684332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:30.815764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:30.817631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5f701-640a-4c19-b942-37efce085b15 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:30.841234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 730/1464] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:32.984759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a080-4dcd-44d1-9802-245c8b5072b1 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:32.985371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a080-4dcd-44d1-9802-245c8b5072b1 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:32.985371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a080-4dcd-44d1-9802-245c8b5072b1 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:32.986710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:32.992528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9832d2-c18b-45c5-81bc-997167ebb006 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:32.993069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9832d2-c18b-45c5-81bc-997167ebb006 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:32.993069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9832d2-c18b-45c5-81bc-997167ebb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:33.064254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5c9832d2-c18b-45c5-81bc-997167ebb006 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:33.065528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5c9832d2-c18b-45c5-81bc-997167ebb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:33.085859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:33.172620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:33.211992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a080-4dcd-44d1-9802-245c8b5072b1 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:33.213457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f524a080-4dcd-44d1-9802-245c8b5072b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:33.320937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a080-4dcd-44d1-9802-245c8b5072b1 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:33.392959 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:33.401128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:33.401772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:33.402045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:33.402319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:33.404003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:33.526847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:33.528319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa764927-4d55-425a-b89d-919bff924cc5 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:33.547903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 736/1467] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:35.711448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33521e73-1a0d-45e7-b1f1-6693466357cc None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:35.712164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33521e73-1a0d-45e7-b1f1-6693466357cc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:35.712164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33521e73-1a0d-45e7-b1f1-6693466357cc None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:35.713528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:35.719378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6adca-39ed-4f99-bd67-f2871f0d7479 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:35.720001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6adca-39ed-4f99-bd67-f2871f0d7479 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:35.720001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6adca-39ed-4f99-bd67-f2871f0d7479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:35.789341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-dcf6adca-39ed-4f99-bd67-f2871f0d7479 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:35.790553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dcf6adca-39ed-4f99-bd67-f2871f0d7479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:35.842014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:35.893415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:46:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:35.935015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33521e73-1a0d-45e7-b1f1-6693466357cc None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:35.936590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-33521e73-1a0d-45e7-b1f1-6693466357cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:36.044088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33521e73-1a0d-45e7-b1f1-6693466357cc None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:36.095154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:36.123077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:36.123733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:36.124014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:36.124286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:36.126774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:36.259881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:36.262246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4085c4-542a-4f29-baa5-b649eba53ea3 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:36.288952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 733/1470] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:45.890925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=22 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:46.284558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=66 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:56.603439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606d30-de21-4c21-9fc8-c81c30f7f036 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:56.603961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606d30-de21-4c21-9fc8-c81c30f7f036 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:56.604081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17606d30-de21-4c21-9fc8-c81c30f7f036 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:56.605378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 738/1471] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:56.610706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8369b0-cdf4-4aec-a278-e486a2d7612e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:56.611272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8369b0-cdf4-4aec-a278-e486a2d7612e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:56.611272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8369b0-cdf4-4aec-a278-e486a2d7612e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:56.679025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ad8369b0-cdf4-4aec-a278-e486a2d7612e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:56.680277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ad8369b0-cdf4-4aec-a278-e486a2d7612e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:56.782557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 734/1472] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:56.867874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5426a-c480-4860-bc0a-e1f206f8778f None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:56.868091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5426a-c480-4860-bc0a-e1f206f8778f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:56.868300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5426a-c480-4860-bc0a-e1f206f8778f None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:56.869662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 739/1473] 10.222.0.86 () {54 vars in 966 bytes} [Wed Jul 20 21:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:56.875607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4320c8c-2d70-41a0-bee5-cc58775e0d2d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:56.875898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4320c8c-2d70-41a0-bee5-cc58775e0d2d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:56.876172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4320c8c-2d70-41a0-bee5-cc58775e0d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:56.953717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e4320c8c-2d70-41a0-bee5-cc58775e0d2d None None] MFA Rules not processed for user `be5097840a23492585d297d35e2fea20`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:56.954973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e4320c8c-2d70-41a0-bee5-cc58775e0d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:56.976382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:57.034729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 735/1474] 10.222.0.86 () {58 vars in 1063 bytes} [Wed Jul 20 21:46:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:57.042564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5426a-c480-4860-bc0a-e1f206f8778f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:57.044097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-53b5426a-c480-4860-bc0a-e1f206f8778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:57.151573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5426a-c480-4860-bc0a-e1f206f8778f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:57.227473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:46:57.228294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:57.228931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:57.228931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:57.230586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:57.289918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:46:57.350447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:46:57.351922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd44d41-8ef5-4a75-9fab-d61fa9666615 service glance] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:46:57.371987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 740/1475] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:46:59.548665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b668b1-ce9b-4c71-8617-fe313cc55cc3 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:59.549241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b668b1-ce9b-4c71-8617-fe313cc55cc3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:59.549241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b668b1-ce9b-4c71-8617-fe313cc55cc3 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:59.550586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 736/1476] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:46:59.555911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc507e-f6f9-44de-9ad7-c0933a6c71ab None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:46:59.556327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc507e-f6f9-44de-9ad7-c0933a6c71ab None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:46:59.556470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc507e-f6f9-44de-9ad7-c0933a6c71ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:46:59.623221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-33dc507e-f6f9-44de-9ad7-c0933a6c71ab None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:46:59.624486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-33dc507e-f6f9-44de-9ad7-c0933a6c71ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:59.721889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 741/1477] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:46:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:46:59.751746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b668b1-ce9b-4c71-8617-fe313cc55cc3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:46:59.754657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-13b668b1-ce9b-4c71-8617-fe313cc55cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:46:59.904132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b668b1-ce9b-4c71-8617-fe313cc55cc3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:46:59.959646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:00.005276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:47:00.006034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:00.006323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:00.006675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:00.007366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:47:00.009187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:47:00.009187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:47:00.009187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:47:00.010337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa537ea-ea78-4ee0-8c92-162ae518b570 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:47:00.030257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 737/1478] 10.222.0.86 () {56 vars in 1108 bytes} [Wed Jul 20 21:46:59 2022] GET /identity/v3/projects => generated 2553 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:47:02.186570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4d314-5c44-4afe-b3d3-55f585a818dd None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:02.187187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4d314-5c44-4afe-b3d3-55f585a818dd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:02.187309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4d314-5c44-4afe-b3d3-55f585a818dd None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:02.188673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 742/1479] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:47:02.194292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f1874-f1eb-4983-ac0a-432d36b6e6a3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:02.194801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f1874-f1eb-4983-ac0a-432d36b6e6a3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:02.194911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f1874-f1eb-4983-ac0a-432d36b6e6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:02.266431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-190f1874-f1eb-4983-ac0a-432d36b6e6a3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:47:02.267687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-190f1874-f1eb-4983-ac0a-432d36b6e6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:02.371964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 738/1480] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:47:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:47:02.431613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4d314-5c44-4afe-b3d3-55f585a818dd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:47:02.433218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9af4d314-5c44-4afe-b3d3-55f585a818dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:02.448163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:02.596932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4d314-5c44-4afe-b3d3-55f585a818dd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:47:02.683611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:02.720442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:47:02.721089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:02.721364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:02.721633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:02.723310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:02.819929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:47:02.821532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae09a8-a17e-4cae-a64c-cf078dcefbdd service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:47:02.840850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 743/1481] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:47:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:47:05.142662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22372fef-0ec3-4a8e-836c-110374633d3f None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:05.143287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22372fef-0ec3-4a8e-836c-110374633d3f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:05.143287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22372fef-0ec3-4a8e-836c-110374633d3f None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:05.144620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 739/1482] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:47:05.150080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c485fdb-2816-4ffb-a8bb-24cfe3c14f2b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:05.150547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c485fdb-2816-4ffb-a8bb-24cfe3c14f2b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:05.150651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c485fdb-2816-4ffb-a8bb-24cfe3c14f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:05.216141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0c485fdb-2816-4ffb-a8bb-24cfe3c14f2b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:47:05.217469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0c485fdb-2816-4ffb-a8bb-24cfe3c14f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:05.265375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:05.316964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 744/1483] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:47:05.353978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22372fef-0ec3-4a8e-836c-110374633d3f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:47:05.355593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-22372fef-0ec3-4a8e-836c-110374633d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:05.381933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:05.473976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22372fef-0ec3-4a8e-836c-110374633d3f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:47:05.558486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:47:05.559212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:05.559490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:05.559762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:05.562112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:05.638546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:05.674734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:47:05.676221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aeb515-54ab-4602-815f-3451970997f4 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:47:05.696195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 740/1484] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:47:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:47:07.858819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6161d7-43ce-4bf0-acc3-c730ed98f3f1 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:07.859503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6161d7-43ce-4bf0-acc3-c730ed98f3f1 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:07.859503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6161d7-43ce-4bf0-acc3-c730ed98f3f1 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:07.860789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 745/1485] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:47:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:47:07.867305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9c5f5-46ad-42da-b68b-37e61d2d7f3b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:07.867795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9c5f5-46ad-42da-b68b-37e61d2d7f3b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:07.867904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9c5f5-46ad-42da-b68b-37e61d2d7f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:07.938404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4ab9c5f5-46ad-42da-b68b-37e61d2d7f3b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:47:07.939685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4ab9c5f5-46ad-42da-b68b-37e61d2d7f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:08.047567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 741/1486] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:47:08.088731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6161d7-43ce-4bf0-acc3-c730ed98f3f1 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:47:08.090400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3b6161d7-43ce-4bf0-acc3-c730ed98f3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:08.213106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6161d7-43ce-4bf0-acc3-c730ed98f3f1 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:47:08.290156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:08.292040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:47:08.292739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:08.293063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:08.293542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:08.296474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:08.413860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:47:08.415312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea7837-e616-4d36-a665-07ac5ac04639 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:47:08.434937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 746/1487] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:47:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:47:10.940720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c438be4-2f4c-4be3-8262-4299a67e899a None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:10.941249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c438be4-2f4c-4be3-8262-4299a67e899a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:10.941249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c438be4-2f4c-4be3-8262-4299a67e899a None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:10.942550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 742/1488] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:47:10.947903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3cbe-14be-4a73-aa14-d591dda30553 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:10.948333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3cbe-14be-4a73-aa14-d591dda30553 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:10.948460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f3cbe-14be-4a73-aa14-d591dda30553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:11.018091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c02f3cbe-14be-4a73-aa14-d591dda30553 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:47:11.019382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c02f3cbe-14be-4a73-aa14-d591dda30553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:11.074342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:11.120840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 747/1489] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:47:11.163548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c438be4-2f4c-4be3-8262-4299a67e899a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:47:11.165127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6c438be4-2f4c-4be3-8262-4299a67e899a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:11.203503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:11.273924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c438be4-2f4c-4be3-8262-4299a67e899a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:47:11.348601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:47:11.349161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:11.349446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:11.349724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:11.351418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:11.449171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:11.482300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:47:11.484589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa53d82-cddb-4764-869c-ce2054c91609 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:47:11.503850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 743/1490] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:47:13.909603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276b27-8ed8-42a7-a111-e9c17c09cd35 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:13.910179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276b27-8ed8-42a7-a111-e9c17c09cd35 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:13.910179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276b27-8ed8-42a7-a111-e9c17c09cd35 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:13.911535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 748/1491] 10.222.0.86 () {54 vars in 926 bytes} [Wed Jul 20 21:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:47:13.916663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed50979-56a9-4c7a-97de-f15b7e6fafa0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:13.917060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed50979-56a9-4c7a-97de-f15b7e6fafa0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:13.917221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed50979-56a9-4c7a-97de-f15b7e6fafa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:13.981837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9ed50979-56a9-4c7a-97de-f15b7e6fafa0 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:47:13.983269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9ed50979-56a9-4c7a-97de-f15b7e6fafa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:14.081365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 744/1492] 10.222.0.86 () {58 vars in 1023 bytes} [Wed Jul 20 21:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:47:14.122397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c276b27-8ed8-42a7-a111-e9c17c09cd35 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:47:14.124869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2c276b27-8ed8-42a7-a111-e9c17c09cd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:14.230819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c276b27-8ed8-42a7-a111-e9c17c09cd35 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:47:14.305418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:47:14.306066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:47:14.306349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:47:14.306626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:47:14.308338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:47:14.318223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:14.419531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:47:14.421551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f50575-a901-4401-83c0-84b8c2980e61 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:47:14.440350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 749/1493] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:47:24.078773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=82 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:47:24.438104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=50 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:43.911391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:43.912002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:43.912002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:43.992826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:48:43.994054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:44.097210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 745/1494] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:48:44.106468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:44.107021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:44.107021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:44.185215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:48:44.186362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:44.288725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 750/1495] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:48:44.294673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:44.295954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:44.307885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd19393-5acb-4c4c-836a-335eff46f083 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:44.374285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:44.410384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:44.411043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:44.411350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:44.411624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:44.412352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:48:44.412662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:44.413682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:44.413682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:48:44.413682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:48:44.414561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:44.425913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 746/1496] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:48:44.434497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:44.444357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a6ddd8-e4cb-44a3-b836-6a436896e899 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:44.506252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:44.542963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:44.543672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:44.543945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:44.544217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:44.544913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:44.546682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:44.546682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:48:44.546682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:48:44.547905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:44.606297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 751/1497] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:48:44 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:48:44.610540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:44.612147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:44.647114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401139-2283-44c6-98fb-1cad16cd447a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:44.743168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:44.743842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:44.744119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:44.744390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:44.745086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:44.746815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:44.746815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:48:44.746815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:48:44.747894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:44.790556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 747/1498] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:48:44 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:48:44.795507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:44.797072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:44.862150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1c7d4-7d13-4107-b98f-8d7ae300655d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:44.951847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:44.964433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:44.965128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:44.965403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:44.965676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:44.966418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:44.967482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:44.967482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:48:44.967482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:48:44.968379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:44.982545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 752/1499] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:48:44.987386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:44.990634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:45.008590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb1c00-e12e-4a2a-ad15-9142f92d7960 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:45.034613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:45.109296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:45.109971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:45.110259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:45.110538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:45.111297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:45.112327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:45.112327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:48:45.112327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:48:45.113217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c12a3-c480-4350-a7a7-c4946aa706d3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:45.126631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 748/1500] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:48:45.130846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:45.132423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:45.166417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfd575-60f1-490a-ba36-10fdbcbf7c69 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:45.264677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:45.265476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:45.265756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:45.266032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] PATH_INFO: `/v3/projects/5094e8466b9b4078adb20d1154a2b5b7/users/42dc18dcdfc749259a55d9adac4193cc/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:45.288613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5094e8466b9b4078adb20d1154a2b5b7, user_id=42dc18dcdfc749259a55d9adac4193cc, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:45.289648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:45.289648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:48:45.289648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:48:45.291627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58ff40-f5d3-43e2-88ff-cec7f0a7ec1b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:45.316874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 753/1501] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:48:45 2022] PUT /identity/v3/projects/5094e8466b9b4078adb20d1154a2b5b7/users/42dc18dcdfc749259a55d9adac4193cc/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:48:45.330210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0558d26-ed03-44e1-86f2-5400605036c6 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:45.330494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0558d26-ed03-44e1-86f2-5400605036c6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:45.330769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0558d26-ed03-44e1-86f2-5400605036c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:45.386245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b0558d26-ed03-44e1-86f2-5400605036c6 None None] MFA Rules not processed for user `42dc18dcdfc749259a55d9adac4193cc`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:48:45.387509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b0558d26-ed03-44e1-86f2-5400605036c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:45.466807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 749/1502] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:48:45.481049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5c231-f46b-49d5-904a-63af0fe5cee2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:45.481334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5c231-f46b-49d5-904a-63af0fe5cee2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:45.481614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5c231-f46b-49d5-904a-63af0fe5cee2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:45.483159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 754/1503] 10.222.0.86 () {54 vars in 964 bytes} [Wed Jul 20 21:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:48:45.488314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4e233-ab09-4c05-a48a-e48b7ddbe09a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:45.488608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4e233-ab09-4c05-a48a-e48b7ddbe09a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:45.488872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4e233-ab09-4c05-a48a-e48b7ddbe09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:45.520482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:45.564922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-96c4e233-ab09-4c05-a48a-e48b7ddbe09a None None] MFA Rules not processed for user `ee043775ebb54bed8642c9f735b46c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:48:45.566146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-96c4e233-ab09-4c05-a48a-e48b7ddbe09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:45.642878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 750/1504] 10.222.0.86 () {58 vars in 1061 bytes} [Wed Jul 20 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:48:45.650488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5c231-f46b-49d5-904a-63af0fe5cee2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:45.652295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b8a5c231-f46b-49d5-904a-63af0fe5cee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:45.702294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:45.760252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5c231-f46b-49d5-904a-63af0fe5cee2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:45.835115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:45.835696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:45.835979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:45.836256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:45.837931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:45.939772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:45.941208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:45.944742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:45.960216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 755/1505] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:48:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:48:46.004326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22251b-6222-4272-bae3-c68016af1781 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:46.004774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22251b-6222-4272-bae3-c68016af1781 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:46.004887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22251b-6222-4272-bae3-c68016af1781 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:46.006149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 751/1506] 10.222.0.86 () {52 vars in 829 bytes} [Wed Jul 20 21:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:48:46.009824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:46.011139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:46.110377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d447d-6a6d-44c6-9f1a-755f571a6637 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:46.208198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:46.209107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:46.209404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:46.209676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] PATH_INFO: `/v3/users/42dc18dcdfc749259a55d9adac4193cc` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:46.225275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42dc18dcdfc749259a55d9adac4193cc)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:46.226316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:46.226316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:48:46.226316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:48:46.227588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:46.240548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:46.303612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42dc18dcdfc749259a55d9adac4193cc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:46.323518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42dc18dcdfc749259a55d9adac4193cc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:46.335863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42dc18dcdfc749259a55d9adac4193cc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:46.346617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42dc18dcdfc749259a55d9adac4193cc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:46.347191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42dc18dcdfc749259a55d9adac4193cc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:46.353572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 756/1507] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:48:46 2022] DELETE /identity/v3/users/42dc18dcdfc749259a55d9adac4193cc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:48:46.358885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:46.359198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:46.359478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:46.433260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:48:46.434383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:46.549437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:46.566162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 752/1508] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:48:46.576405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:46.577667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:46.706339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f0a04-9768-4bd0-a73d-c69e907a686c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:46.763891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:46.787357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:46.788026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:46.788312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:46.788590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:46.790372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:46.933726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:46.935234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c78ab1-2122-42ee-a9f9-d5e2ec97ee64 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:46.958418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 757/1509] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:48:47.449299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:48:47.451666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:48:47.466352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fa11e-170c-4d22-8aca-da30321c045f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:48:47.596519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:48:47.597302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:48:47.597583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:48:47.597857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] PATH_INFO: `/v3/projects/5094e8466b9b4078adb20d1154a2b5b7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:48:47.609022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5094e8466b9b4078adb20d1154a2b5b7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:48:47.610071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:48:47.610071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:48:47.610071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:48:47.611320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:48:47.693065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5094e8466b9b4078adb20d1154a2b5b7', 'request_id': 'req-3dc5e497-ad77-4171-84d8-5f614ad0be8b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:47.707204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5094e8466b9b4078adb20d1154a2b5b7', 'request_id': 'req-3dc5e497-ad77-4171-84d8-5f614ad0be8b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:47.713426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5094e8466b9b4078adb20d1154a2b5b7', 'request_id': 'req-3dc5e497-ad77-4171-84d8-5f614ad0be8b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:47.742748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] The token cache is being invalidate because project 5094e8466b9b4078adb20d1154a2b5b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:48:47.743033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:47.743609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3dc5e497-ad77-4171-84d8-5f614ad0be8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:48:47.745003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 753/1510] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:48:47 2022] DELETE /identity/v3/projects/5094e8466b9b4078adb20d1154a2b5b7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:48:56.955616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=62 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:48:57.740600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:50:33.475520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf4563-2442-4e6a-898a-23fc66910cc2 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:33.476122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf4563-2442-4e6a-898a-23fc66910cc2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:33.476122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf4563-2442-4e6a-898a-23fc66910cc2 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:33.478269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:33.485517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5b9c-8a14-4c2e-a734-9f599cac88fb None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:33.487691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5b9c-8a14-4c2e-a734-9f599cac88fb None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:33.487691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e5b9c-8a14-4c2e-a734-9f599cac88fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:33.580267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a43e5b9c-8a14-4c2e-a734-9f599cac88fb None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:33.582183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a43e5b9c-8a14-4c2e-a734-9f599cac88fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:33.700181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:33.860462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf4563-2442-4e6a-898a-23fc66910cc2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:33.862684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7ebf4563-2442-4e6a-898a-23fc66910cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:34.001908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf4563-2442-4e6a-898a-23fc66910cc2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:34.007659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55ac28-5c4d-4c15-ac61-1b05c33c4fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:34.008974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55ac28-5c4d-4c15-ac61-1b05c33c4fbb service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:34.009257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55ac28-5c4d-4c15-ac61-1b05c33c4fbb service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:34.009524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55ac28-5c4d-4c15-ac61-1b05c33c4fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:34.018361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55ac28-5c4d-4c15-ac61-1b05c33c4fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:34.019990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55ac28-5c4d-4c15-ac61-1b05c33c4fbb service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:34.030320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 759/1513] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:50:34.899688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a32881-97a4-43bb-b048-bfe7dbb9ff7b None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:34.900275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a32881-97a4-43bb-b048-bfe7dbb9ff7b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:34.900534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a32881-97a4-43bb-b048-bfe7dbb9ff7b None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:34.902760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:50:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:34.908144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd43888-81a2-40ea-954d-ce281b988d7f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:34.908778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd43888-81a2-40ea-954d-ce281b988d7f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:34.908778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd43888-81a2-40ea-954d-ce281b988d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:34.928039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fbd43888-81a2-40ea-954d-ce281b988d7f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:34.929308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fbd43888-81a2-40ea-954d-ce281b988d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:34.938164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:35.094407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a32881-97a4-43bb-b048-bfe7dbb9ff7b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:35.112443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a32881-97a4-43bb-b048-bfe7dbb9ff7b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:35.119125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa17fdb-e3fb-47b3-bcf8-b328c886cbe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:35.119804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa17fdb-e3fb-47b3-bcf8-b328c886cbe0 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:35.120074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa17fdb-e3fb-47b3-bcf8-b328c886cbe0 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:35.120352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa17fdb-e3fb-47b3-bcf8-b328c886cbe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:35.130505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa17fdb-e3fb-47b3-bcf8-b328c886cbe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:35.132153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa17fdb-e3fb-47b3-bcf8-b328c886cbe0 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:35.143127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 756/1516] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:50:37.251554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f989-73d5-4cb4-b7d7-6acb1b4ce01a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:37.252124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f989-73d5-4cb4-b7d7-6acb1b4ce01a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:37.252124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f989-73d5-4cb4-b7d7-6acb1b4ce01a None None] PATH_INFO: `/v3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:37.253542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 761/1517] 10.222.0.74 () {54 vars in 925 bytes} [Wed Jul 20 21:50:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:50:37.261004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d811a-51d9-4697-9c04-bfd7c4262f26 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:37.261477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d811a-51d9-4697-9c04-bfd7c4262f26 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:37.261597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d811a-51d9-4697-9c04-bfd7c4262f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:37.335106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-007d811a-51d9-4697-9c04-bfd7c4262f26 None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:37.336461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-007d811a-51d9-4697-9c04-bfd7c4262f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:37.357148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:50:37.419664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 757/1518] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Jul 20 21:50:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:41.704523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95da13-7856-4acf-8700-50f5fec80825 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:41.705081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95da13-7856-4acf-8700-50f5fec80825 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:41.705081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95da13-7856-4acf-8700-50f5fec80825 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:41.706354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 762/1519] 10.222.0.86 () {54 vars in 961 bytes} [Wed Jul 20 21:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:41.712694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe46ee-7abb-45d5-aa79-d4912fcc3680 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:41.713181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe46ee-7abb-45d5-aa79-d4912fcc3680 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:41.713292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe46ee-7abb-45d5-aa79-d4912fcc3680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:41.731577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-dafe46ee-7abb-45d5-aa79-d4912fcc3680 None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:41.732781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dafe46ee-7abb-45d5-aa79-d4912fcc3680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:41.742852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 758/1520] 10.222.0.86 () {58 vars in 1058 bytes} [Wed Jul 20 21:50:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:41.750540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95da13-7856-4acf-8700-50f5fec80825 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:41.765079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95da13-7856-4acf-8700-50f5fec80825 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:41.770193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b632e6-20df-4515-acd9-019ae28bfba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:41.770843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b632e6-20df-4515-acd9-019ae28bfba9 service placement] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:41.771163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b632e6-20df-4515-acd9-019ae28bfba9 service placement] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:41.771445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b632e6-20df-4515-acd9-019ae28bfba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:41.781321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b632e6-20df-4515-acd9-019ae28bfba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:41.782914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b632e6-20df-4515-acd9-019ae28bfba9 service placement] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:41.793643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 763/1521] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:50:46.410561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8237fd-75c0-4920-b527-f7954d8c7b0c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:46.411123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8237fd-75c0-4920-b527-f7954d8c7b0c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:46.411123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8237fd-75c0-4920-b527-f7954d8c7b0c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:46.412449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:46.417721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb0315-cd1d-4801-ae0c-22fc1fc81cd5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:46.418253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb0315-cd1d-4801-ae0c-22fc1fc81cd5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:46.418253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb0315-cd1d-4801-ae0c-22fc1fc81cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:46.438181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b3cb0315-cd1d-4801-ae0c-22fc1fc81cd5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:46.439495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b3cb0315-cd1d-4801-ae0c-22fc1fc81cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:46.448634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:50:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:50:46.613534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8237fd-75c0-4920-b527-f7954d8c7b0c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:46.629919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8237fd-75c0-4920-b527-f7954d8c7b0c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:46.636723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae8b03-fa0b-4b4f-821e-379debf1ea57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:46.637400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae8b03-fa0b-4b4f-821e-379debf1ea57 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:46.637705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae8b03-fa0b-4b4f-821e-379debf1ea57 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:46.637991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae8b03-fa0b-4b4f-821e-379debf1ea57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:46.647224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae8b03-fa0b-4b4f-821e-379debf1ea57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:46.649345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae8b03-fa0b-4b4f-821e-379debf1ea57 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:46.659800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 760/1524] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:50:47.529625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101621ad-201e-4ce4-bca9-f1743c87663b None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:47.530226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101621ad-201e-4ce4-bca9-f1743c87663b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:47.530226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101621ad-201e-4ce4-bca9-f1743c87663b None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:47.531589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 21:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:47.536695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd93f8-b7e1-49a1-b041-e9be5f3966d2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:47.537427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd93f8-b7e1-49a1-b041-e9be5f3966d2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:47.537427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd93f8-b7e1-49a1-b041-e9be5f3966d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:47.556603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-14cd93f8-b7e1-49a1-b041-e9be5f3966d2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:47.557820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-14cd93f8-b7e1-49a1-b041-e9be5f3966d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:47.567604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 21:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:47.728069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101621ad-201e-4ce4-bca9-f1743c87663b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:47.743810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101621ad-201e-4ce4-bca9-f1743c87663b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:47.749546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec7c31-45a1-46f2-a1e3-6e52beb0f0c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:47.750201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec7c31-45a1-46f2-a1e3-6e52beb0f0c1 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:47.750484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec7c31-45a1-46f2-a1e3-6e52beb0f0c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:47.750761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec7c31-45a1-46f2-a1e3-6e52beb0f0c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:47.760608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec7c31-45a1-46f2-a1e3-6e52beb0f0c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:47.762211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec7c31-45a1-46f2-a1e3-6e52beb0f0c1 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:47.772639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 766/1527] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:50:51.966047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55e253-33a7-4444-a41a-a2e8b8cd1c8f None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:51.966638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55e253-33a7-4444-a41a-a2e8b8cd1c8f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:51.966638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55e253-33a7-4444-a41a-a2e8b8cd1c8f None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:51.968008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:51.974376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc3ec-914e-43b8-86cc-2d156071caaf None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:51.974965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc3ec-914e-43b8-86cc-2d156071caaf None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:51.974965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbdc3ec-914e-43b8-86cc-2d156071caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:51.994679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-7cbdc3ec-914e-43b8-86cc-2d156071caaf None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:51.995926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7cbdc3ec-914e-43b8-86cc-2d156071caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:52.004115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:52.094024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55e253-33a7-4444-a41a-a2e8b8cd1c8f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:52.095911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fb55e253-33a7-4444-a41a-a2e8b8cd1c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:52.193919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55e253-33a7-4444-a41a-a2e8b8cd1c8f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:52.197979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97daacfc-1ba6-4be5-bbd7-b5ec35b759b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:52.198644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daacfc-1ba6-4be5-bbd7-b5ec35b759b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:52.198925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daacfc-1ba6-4be5-bbd7-b5ec35b759b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:52.199235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97daacfc-1ba6-4be5-bbd7-b5ec35b759b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:52.209255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daacfc-1ba6-4be5-bbd7-b5ec35b759b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:52.210847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97daacfc-1ba6-4be5-bbd7-b5ec35b759b3 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:52.221776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 763/1530] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:50:54.452437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f802a4-3a69-4181-aefb-d6f2d6ae65eb None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:54.453001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f802a4-3a69-4181-aefb-d6f2d6ae65eb None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:54.453001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f802a4-3a69-4181-aefb-d6f2d6ae65eb None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:54.454298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:54.459742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b4d85-797d-4646-bb13-b88a52fdb5b5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:54.460221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b4d85-797d-4646-bb13-b88a52fdb5b5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:54.460399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b4d85-797d-4646-bb13-b88a52fdb5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:54.480612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-246b4d85-797d-4646-bb13-b88a52fdb5b5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:54.481846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-246b4d85-797d-4646-bb13-b88a52fdb5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:54.489936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:54.528605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f802a4-3a69-4181-aefb-d6f2d6ae65eb None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:54.540401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f802a4-3a69-4181-aefb-d6f2d6ae65eb None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:54.543969 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc6bb0-317d-4341-bd86-858d73c6ca19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:54.544624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc6bb0-317d-4341-bd86-858d73c6ca19 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:54.544903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc6bb0-317d-4341-bd86-858d73c6ca19 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:54.545174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc6bb0-317d-4341-bd86-858d73c6ca19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:54.557610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc6bb0-317d-4341-bd86-858d73c6ca19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:54.559089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc6bb0-317d-4341-bd86-858d73c6ca19 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:54.569570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 769/1533] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:50:58.208113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132aed-010d-4825-98b3-97c157202565 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:58.208671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132aed-010d-4825-98b3-97c157202565 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:58.208671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80132aed-010d-4825-98b3-97c157202565 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:58.210029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:50:58.215472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bb4a2-73a7-4cdb-b82c-3dbfb7fa0f6e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:58.215977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bb4a2-73a7-4cdb-b82c-3dbfb7fa0f6e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:58.216083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830bb4a2-73a7-4cdb-b82c-3dbfb7fa0f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:58.222345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=81 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:50:58.235981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-830bb4a2-73a7-4cdb-b82c-3dbfb7fa0f6e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:50:58.237299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-830bb4a2-73a7-4cdb-b82c-3dbfb7fa0f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:58.246499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:50:58.566638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132aed-010d-4825-98b3-97c157202565 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:50:58.568363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-80132aed-010d-4825-98b3-97c157202565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:50:58.637917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:50:58.653062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80132aed-010d-4825-98b3-97c157202565 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:50:58.656807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1543b-15f6-4f2d-b526-a5017b0e633f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:50:58.657472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1543b-15f6-4f2d-b526-a5017b0e633f service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:50:58.657747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1543b-15f6-4f2d-b526-a5017b0e633f service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:50:58.658021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1543b-15f6-4f2d-b526-a5017b0e633f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:50:58.668560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1543b-15f6-4f2d-b526-a5017b0e633f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:50:58.670014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1543b-15f6-4f2d-b526-a5017b0e633f service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:50:58.680431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 766/1536] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:50:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:51:00.779189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e010a-021f-4d8e-81ad-6c407e159b5a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:51:00.779765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e010a-021f-4d8e-81ad-6c407e159b5a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:51:00.779765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e010a-021f-4d8e-81ad-6c407e159b5a None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:51:00.781167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 21:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:51:00.786701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29041156-023f-4c74-8ccd-6bad37bd3e4c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:51:00.787194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29041156-023f-4c74-8ccd-6bad37bd3e4c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:51:00.787358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29041156-023f-4c74-8ccd-6bad37bd3e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:51:00.806236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-29041156-023f-4c74-8ccd-6bad37bd3e4c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:51:00.807508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-29041156-023f-4c74-8ccd-6bad37bd3e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:51:00.817250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 21:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:51:01.133125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e010a-021f-4d8e-81ad-6c407e159b5a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:51:01.134855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ab7e010a-021f-4d8e-81ad-6c407e159b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:51:01.150141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e010a-021f-4d8e-81ad-6c407e159b5a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:51:01.153721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0175a5-188f-4565-b3ad-f1ed89a821a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:51:01.154366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0175a5-188f-4565-b3ad-f1ed89a821a6 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:51:01.154643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0175a5-188f-4565-b3ad-f1ed89a821a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:51:01.154912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0175a5-188f-4565-b3ad-f1ed89a821a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:51:01.165104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0175a5-188f-4565-b3ad-f1ed89a821a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:51:01.166520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0175a5-188f-4565-b3ad-f1ed89a821a6 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:51:01.176155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 772/1539] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:51:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:51:10.800646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=11 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:51:11.171829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:33.676715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.677655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.677655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.704079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:33.705433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:33.719189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 768/1540] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:33.729288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b8eee-0626-4e28-b343-98af3ebb60b5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.730712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b8eee-0626-4e28-b343-98af3ebb60b5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.731458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b8eee-0626-4e28-b343-98af3ebb60b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.775807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-273b8eee-0626-4e28-b343-98af3ebb60b5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:33.778154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-273b8eee-0626-4e28-b343-98af3ebb60b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:33.797212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 773/1541] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:33.804316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:33.830761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7120615-2f1d-4ed2-96f1-343e2907b6c2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:33.843578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:33.844269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.844546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.844818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.845559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:33.845864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:33.847858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22040a4-941c-4624-b5c6-72f5f0990ddd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:33.858164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2799-eea2-4fe8-8893-8647bfd9f873 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.859487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2799-eea2-4fe8-8893-8647bfd9f873 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.860275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b2799-eea2-4fe8-8893-8647bfd9f873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.869418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 769/1542] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:33.870758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.871098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.871440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.904935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-5a0b2799-eea2-4fe8-8893-8647bfd9f873 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:33.906618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5a0b2799-eea2-4fe8-8893-8647bfd9f873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:33.913185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:33.914448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:33.930837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 774/1543] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:33.933118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66b90-b16b-402f-a500-ea3a58a6294a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.933647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66b90-b16b-402f-a500-ea3a58a6294a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.934215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66b90-b16b-402f-a500-ea3a58a6294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.939152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 770/1544] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:33.939152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:33.969495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8039c-f9a2-4e2c-91e5-ab13265a65ff None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:33.975498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-cad66b90-b16b-402f-a500-ea3a58a6294a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:33.977547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-cad66b90-b16b-402f-a500-ea3a58a6294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:33.988048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:33.988978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.989619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.989779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:33.991475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:33.993439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:52:33.993439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:52:33.993439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:52:33.994685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7cf7e-9ad0-4bb4-a7e5-d14b82a82a0a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:33.995678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 775/1545] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:33.997431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd45c7f-6428-40fb-af51-ef655372d4f8 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:33.997829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd45c7f-6428-40fb-af51-ef655372d4f8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:33.998229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd45c7f-6428-40fb-af51-ef655372d4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.027644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-6fd45c7f-6428-40fb-af51-ef655372d4f8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.029061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6fd45c7f-6428-40fb-af51-ef655372d4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.044315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 771/1546] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.045006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 776/1547] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.045590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e127ca-0baa-46b0-9697-622a69d8368a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.045878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e127ca-0baa-46b0-9697-622a69d8368a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.046156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e127ca-0baa-46b0-9697-622a69d8368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.050435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0ec15-9e51-4cbd-b7e9-b3eeb8f555d5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.050728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0ec15-9e51-4cbd-b7e9-b3eeb8f555d5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.051124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0ec15-9e51-4cbd-b7e9-b3eeb8f555d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.070414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-00e127ca-0baa-46b0-9697-622a69d8368a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.071869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-00e127ca-0baa-46b0-9697-622a69d8368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.074204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-23b0ec15-9e51-4cbd-b7e9-b3eeb8f555d5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.076360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-23b0ec15-9e51-4cbd-b7e9-b3eeb8f555d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.141918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 772/1548] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.143385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e6221-5ede-418d-8140-ada1eeb8ffce None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.143608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e6221-5ede-418d-8140-ada1eeb8ffce None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.143840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e6221-5ede-418d-8140-ada1eeb8ffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.153480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 777/1549] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.154787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.155097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.155375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.164642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8b3e6221-5ede-418d-8140-ada1eeb8ffce None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.165997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8b3e6221-5ede-418d-8140-ada1eeb8ffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.174968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 773/1550] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.176290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.176573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.176850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.180167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.181496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.190167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 778/1551] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.190880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.197781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.199002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.200706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6f06b-4c3f-4538-9c18-c8d3a28f0318 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.205490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.206168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.206452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.206730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.207267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 774/1552] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.207488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.208157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.208579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:52:34.208579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:52:34.208579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:52:34.210395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35e7a0-342c-46dc-b396-3872517eb7bd admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.221312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4abbf-98c2-43f6-a4f2-12d408546614 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.227194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.227877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.228160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.228442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.229207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.229517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.231168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.233498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 779/1553] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.234884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb4465-2950-43c5-98dc-d4aaa7bca817 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.235212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb4465-2950-43c5-98dc-d4aaa7bca817 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.235494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb4465-2950-43c5-98dc-d4aaa7bca817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.248582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 775/1554] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.248582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.256034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c4bb4465-2950-43c5-98dc-d4aaa7bca817 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.256123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c4bb4465-2950-43c5-98dc-d4aaa7bca817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.261172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2d5c6-9e6b-450a-856c-495bc4dadcd4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.271079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 780/1555] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.271425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.272738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.272963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.273092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.273258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.273459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.274085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.274201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.274512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.276371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8ed58-6ee3-4a24-a74c-33116e08197c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.291832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 776/1556] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.293526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.294753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.296275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270bb60-951d-4d93-9544-55e4cdc92781 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.296662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270bb60-951d-4d93-9544-55e4cdc92781 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.296942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270bb60-951d-4d93-9544-55e4cdc92781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.303120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 781/1557] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.305163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.314351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f9c85-a12c-4f6a-986e-7e6bc45e3e56 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.315302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d270bb60-951d-4d93-9544-55e4cdc92781 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.316523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d270bb60-951d-4d93-9544-55e4cdc92781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.319277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.320047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.320362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.320643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.321386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.321697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.322742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:52:34.322742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:52:34.322742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:52:34.323697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.324973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 777/1558] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.326317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.326602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.326882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.344311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 782/1559] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.344311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.348673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb4feb-1de6-436c-a239-5981d7c41b6b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.351196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.352489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.354279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35ad85-6bd8-4880-976d-ea45ecf858f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.354960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35ad85-6bd8-4880-976d-ea45ecf858f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.355275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35ad85-6bd8-4880-976d-ea45ecf858f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.355553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35ad85-6bd8-4880-976d-ea45ecf858f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.356294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35ad85-6bd8-4880-976d-ea45ecf858f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.357893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35ad85-6bd8-4880-976d-ea45ecf858f4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.360453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 778/1560] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.361309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.373659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 783/1561] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:34.373922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efc4ed-998d-4ab7-a27e-a94578bb266d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.374026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.374026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.374026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.376887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.377546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.377824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.378092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.378776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.380727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.390852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:34.392125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:34.405571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 784/1562] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:34.410116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.410875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 779/1563] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.411732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.420946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaced2-4320-4a24-9a2f-fb1ad8e1ab23 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.446161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:34.480004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333d2f-bf9e-4bd3-9d27-a193929e957e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.480703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.481379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.481664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.481943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.482645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.484595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.484734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.485394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.485681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.485957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.486689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.487001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.488633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.498219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:34.502580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 780/1564] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.503656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.515725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34548f26-a6c2-4c96-b64b-e61c9c32cfaa admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.522586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 785/1565] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.522586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.522818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.522818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.522818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.522818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.523810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.524121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.525733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.539273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 781/1566] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.540118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.550137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e714df9-d73b-49a1-8ef6-0a4b42fff79f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.588709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ccaa-d2e4-4218-9bd8-f64335a7c7b2 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.593403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.594066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.594377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.594655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.595393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.597305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.608782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.609460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.609746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.610026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.610765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.611099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.612737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.624992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 786/1567] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.632870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 782/1568] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.632870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.633160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.637658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2f7ad-43be-48a4-ac18-b0894bbe1a8e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.692829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af07d6-c7d2-4583-8fee-8bb2b35e4af5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.697566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.697841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.698314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.698408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.698495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.698761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.698865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.699122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.699643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.699863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:34.700173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.701242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.701771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.712719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 787/1569] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:34.722381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 783/1570] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:34.722613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.722718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.728100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b6e08-c9e2-4398-bfa7-f6244d0686ea admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.728809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83485e9-cb39-4e70-be40-a9fa4b437a0e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.732730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.733424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.733623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.733736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.733948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.734275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.734555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.734665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.734826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.735546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.736594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.737482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.762680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 788/1571] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.762680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.767730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 784/1572] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.768545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.771365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79dc57-82ce-489c-811c-3aea15bd858b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.790926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:34.827704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:34.836822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.837523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.837791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.838063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.838774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.840617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09284f-a89f-4093-894f-9dcdce73c43a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.840954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.845360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.846019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.846298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.846570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.847320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.849399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.865772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 789/1573] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.866454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.877325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cd13e-4234-4f48-9239-8438197a76a4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.882743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.883412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.883656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 785/1574] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/projects => generated 391 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.883834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.884019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:34.884444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.884640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:34.886737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:34.908475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 790/1575] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:34.909387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:34.920703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8ad35-05f3-4836-9491-93b5be338309 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:34.987140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:34.988077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:34.988360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:34.988637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] PATH_INFO: `/v3/projects/a41b592357304d8787cc8e1ab4c9df15/users/c58389cb50e140d69f1d4c99d6d0e182/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.006953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41b592357304d8787cc8e1ab4c9df15, user_id=c58389cb50e140d69f1d4c99d6d0e182, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.009732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.025745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 791/1576] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:34 2022] PUT /identity/v3/projects/a41b592357304d8787cc8e1ab4c9df15/users/c58389cb50e140d69f1d4c99d6d0e182/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:35.030535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.076996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcb492-92e3-44ef-848c-3ff55fbfe88f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.103646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c50455-2ca5-4b7f-8dc8-b7a66899fae5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.109465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.110239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.110635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.110973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.111876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.113928 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.145758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 792/1577] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:35.145758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.148975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.149654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.149989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.150269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.151126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.153057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.182019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 786/1578] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:34 2022] POST /identity/v3/projects => generated 393 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:35.183107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.186907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:35.212807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc0bf1-5b03-4017-820f-6afde61db8ed admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.218746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:35.247428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46f9df-3361-40f0-8194-f40bc13345df admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.252123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.253026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.253312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.253617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.254290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.256814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.274752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.275497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.275780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.276054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.276867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.278395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.280393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 787/1579] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:35.281038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.290962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348a004-e083-40a2-9758-8ac95589827f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.292129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 793/1580] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.293056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.296205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.296876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.297151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.297441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.298176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.299845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.303584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e618d6-9ca3-4038-9c01-4a3f77b834ae admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.308613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55de6f-3003-4160-a62f-777a91f39a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.309329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55de6f-3003-4160-a62f-777a91f39a49 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.309616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55de6f-3003-4160-a62f-777a91f39a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.309896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55de6f-3003-4160-a62f-777a91f39a49 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.310593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55de6f-3003-4160-a62f-777a91f39a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.312027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 788/1581] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.315316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a55de6f-3003-4160-a62f-777a91f39a49 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.315416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.326925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7b4bb-e91b-4e35-89d7-d864ea3787fa admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.333328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.333995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.334306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.334573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.335558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.335950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 794/1582] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:35.337562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.340312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448da606-7000-48da-9462-38f8c774f710 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.340606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448da606-7000-48da-9462-38f8c774f710 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.340887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448da606-7000-48da-9462-38f8c774f710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.351344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 789/1583] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.351344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.358880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-448da606-7000-48da-9462-38f8c774f710 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:35.359640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143fc69-2b09-4dfd-ba09-5821b85fb51e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.360183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-448da606-7000-48da-9462-38f8c774f710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:35.364323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.364996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.365279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.366049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.366752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.368972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.369457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 795/1584] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:35.370059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448da606-7000-48da-9462-38f8c774f710 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.380029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448da606-7000-48da-9462-38f8c774f710 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.386782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.388475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.388900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.389272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.389893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.392077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.394807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 790/1585] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:35.395683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.406811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae2172-db08-4d74-b720-7a528e2dbcc2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.412292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.413285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.413628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.414074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.414984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.416642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.422409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 796/1586] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/users => generated 297 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:35.423546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.431392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 791/1587] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.432247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.437072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1901da-f23a-48b8-ad8b-2b53e6255982 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.442111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.442268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cafa71-8523-487f-a0cc-8f81c6017974 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.442775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.443088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.443359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.444688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.446288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.457401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.458091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.458407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.458659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.459905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.461592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.470547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 797/1588] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.470547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.480942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 792/1589] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.480942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.482172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8a68-bca6-486d-affa-1996241ee9d8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.487958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.488642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.488765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b99d8-2795-4c59-8c2a-77b299e2e429 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.488921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.489193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.489923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.492102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.494127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.494793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.495107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.495385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.496121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.497802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.509405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 798/1590] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.511355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.514067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:35.518225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 793/1591] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.519516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.535987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced8f15-a3b3-46b3-b7fc-d1fa04d9d4ac admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.542985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.543693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.543982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.544262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.544997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.546725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.558478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 794/1592] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.559728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.571433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531f19e-b0e5-404b-94da-ccfa7c4c5ae3 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.576114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.576790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.577074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.577348 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.578089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.579784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.581387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f7c0a-5ac7-4f6c-abc2-bc9b9492a9ec admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.585896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.586565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.586836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.587259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.595341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 795/1593] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.596396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.597218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.598606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.606463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38cf919-50f6-4e6b-b3b4-2e0acd96ee00 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.611047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.611744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.612024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.612299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.613032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.614635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.616817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 799/1594] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:35.618062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.629461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 796/1595] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.629461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.629716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa99ee0-75d8-497d-a277-2c0ab50d3322 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.635270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.636384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.636673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.636950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] PATH_INFO: `/v3/projects/8a1f1bb699504cb38e176e5dabf3040e/users/69a5796d5cbf4c6a9a8de1c9aa9c6ccf/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.638959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b79d2-4373-437c-bde1-cd137642d3c0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.646571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.647974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.648426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.649258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] PATH_INFO: `/v3/projects/37e4a1ceb7b94c4ea11fbbeb8ee0c006/users/fb27ee14314f42cc87e8e6bac2031c70/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.657425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a1f1bb699504cb38e176e5dabf3040e, user_id=69a5796d5cbf4c6a9a8de1c9aa9c6ccf, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.660282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.663398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:35.671989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e4a1ceb7b94c4ea11fbbeb8ee0c006, user_id=fb27ee14314f42cc87e8e6bac2031c70, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.673563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:52:35.673563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:52:35.673563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:52:35.676152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.679993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 800/1596] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:35 2022] PUT /identity/v3/projects/8a1f1bb699504cb38e176e5dabf3040e/users/69a5796d5cbf4c6a9a8de1c9aa9c6ccf/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:35.682877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.696305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 797/1597] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:35 2022] PUT /identity/v3/projects/37e4a1ceb7b94c4ea11fbbeb8ee0c006/users/fb27ee14314f42cc87e8e6bac2031c70/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:35.697810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.761977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:35.766112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaad655-e587-417d-a0b1-ba4e7d13fc16 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.771192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.771929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.772345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.772815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.773578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.775498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.775882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23809d76-b99d-4e4d-a1f3-8712ebdb94bf admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.783400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.784422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.784719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.785446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] PATH_INFO: `/v3/projects/e722bb68652c48f588d8b53351c6f2a6/users/60f7f43c067e4b819aeb488e9f09197a/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.787053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 801/1598] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.788496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.799437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdbc2a-4895-4042-aa72-9acaa624ffc0 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.807552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.808247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.808655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.808837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.809541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.810044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e722bb68652c48f588d8b53351c6f2a6, user_id=60f7f43c067e4b819aeb488e9f09197a, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.811181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.813033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.832148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 802/1599] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.832148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.841526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd515e0c-babd-4ac7-9c91-e9c8b41f4802 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.845098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 798/1600] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:35 2022] PUT /identity/v3/projects/e722bb68652c48f588d8b53351c6f2a6/users/60f7f43c067e4b819aeb488e9f09197a/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:35.849576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:35.920046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5471b-4aad-4e20-a9e0-6d1af6e8c15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.921082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5471b-4aad-4e20-a9e0-6d1af6e8c15c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.921373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5471b-4aad-4e20-a9e0-6d1af6e8c15c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.921652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5471b-4aad-4e20-a9e0-6d1af6e8c15c admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.922388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5471b-4aad-4e20-a9e0-6d1af6e8c15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.924447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5471b-4aad-4e20-a9e0-6d1af6e8c15c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:35.939041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 803/1601] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:35.940546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.940836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.941111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.948538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d451fe-2c8a-47fc-8425-ebdcc6713e93 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:35.954755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd533b3-76e8-4d8b-99e5-a52c6782a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:35.957098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd533b3-76e8-4d8b-99e5-a52c6782a562 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:35.957591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd533b3-76e8-4d8b-99e5-a52c6782a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:35.958512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd533b3-76e8-4d8b-99e5-a52c6782a562 admin admin] PATH_INFO: `/v3/projects/93b13ad4193f4aecb19e375a1cb82516/users/65eed315b2f249e79a6a27b97a55102d/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:35.972849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] MFA Rules not processed for user `69a5796d5cbf4c6a9a8de1c9aa9c6ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:35.974071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:35.984692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd533b3-76e8-4d8b-99e5-a52c6782a562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b13ad4193f4aecb19e375a1cb82516, user_id=65eed315b2f249e79a6a27b97a55102d, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:35.987950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd533b3-76e8-4d8b-99e5-a52c6782a562 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.008764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 799/1602] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:35 2022] PUT /identity/v3/projects/93b13ad4193f4aecb19e375a1cb82516/users/65eed315b2f249e79a6a27b97a55102d/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:36.013084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.013642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.014228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.042975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.044263 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.082628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 804/1603] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.087113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.110527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:36.120423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761a11-424a-474e-96fc-17aef3b644af None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:36.138085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 800/1604] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.138646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.149846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36394729-5886-4f39-b332-6a60ac718e12 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:36.154901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b22a2-269a-4e74-8ce8-63ff4e59aaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:36.155806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b22a2-269a-4e74-8ce8-63ff4e59aaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.156085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b22a2-269a-4e74-8ce8-63ff4e59aaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.156357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b22a2-269a-4e74-8ce8-63ff4e59aaf8 admin admin] PATH_INFO: `/v3/projects/932a50a36025440583c478450e42f9e8/users/74e755a5fe9f4c5c95376beae691381f/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.174814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b22a2-269a-4e74-8ce8-63ff4e59aaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932a50a36025440583c478450e42f9e8, user_id=74e755a5fe9f4c5c95376beae691381f, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:36.178900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b22a2-269a-4e74-8ce8-63ff4e59aaf8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.197627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 801/1605] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:36 2022] PUT /identity/v3/projects/932a50a36025440583c478450e42f9e8/users/74e755a5fe9f4c5c95376beae691381f/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:36.199319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f9a40-d02f-4ac5-87b1-6067db3419c6 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.199619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f9a40-d02f-4ac5-87b1-6067db3419c6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.200327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f9a40-d02f-4ac5-87b1-6067db3419c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.207563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:36.208872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.210627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.210895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] PATH_INFO: `/v3/projects/80320a3595ce4d4cb7c468b0fbec6580/users/1a0d4ce0ac4d47c8a184ea0291461379/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.222155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a54f9a40-d02f-4ac5-87b1-6067db3419c6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.223518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a54f9a40-d02f-4ac5-87b1-6067db3419c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.231603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80320a3595ce4d4cb7c468b0fbec6580, user_id=1a0d4ce0ac4d47c8a184ea0291461379, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:36.234320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.250510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 805/1606] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:36 2022] PUT /identity/v3/projects/80320a3595ce4d4cb7c468b0fbec6580/users/1a0d4ce0ac4d47c8a184ea0291461379/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:36.251505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.278036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:36.333488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83115c4-e640-4e07-8047-1ff48c564784 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:36.339247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 802/1607] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.339499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb0123-4f26-4f47-9b46-1dadea1c7d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:36.340655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb0123-4f26-4f47-9b46-1dadea1c7d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.340815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb0123-4f26-4f47-9b46-1dadea1c7d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.340939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcb1ab-b466-4979-ac2a-77d41a838794 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.341060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb0123-4f26-4f47-9b46-1dadea1c7d9c admin admin] PATH_INFO: `/v3/projects/99ddb19705984622843d5020633d6c31/users/ebdc4f869011444b816472914ab73dd7/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.341195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcb1ab-b466-4979-ac2a-77d41a838794 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.341446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcb1ab-b466-4979-ac2a-77d41a838794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.362634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb0123-4f26-4f47-9b46-1dadea1c7d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ddb19705984622843d5020633d6c31, user_id=ebdc4f869011444b816472914ab73dd7, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:36.365718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb0123-4f26-4f47-9b46-1dadea1c7d9c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.387659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-6efcb1ab-b466-4979-ac2a-77d41a838794 None None] MFA Rules not processed for user `65eed315b2f249e79a6a27b97a55102d`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.389217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6efcb1ab-b466-4979-ac2a-77d41a838794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.394933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 806/1608] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:36 2022] PUT /identity/v3/projects/99ddb19705984622843d5020633d6c31/users/ebdc4f869011444b816472914ab73dd7/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:36.396352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d5212-5555-4f0a-9cf5-98b0a9e9d8db None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.396644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d5212-5555-4f0a-9cf5-98b0a9e9d8db None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.397513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d5212-5555-4f0a-9cf5-98b0a9e9d8db None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.399674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 807/1609] 10.222.0.86 () {54 vars in 962 bytes} [Wed Jul 20 21:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:52:36.400448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d5212-5555-4f0a-9cf5-98b0a9e9d8db None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.514610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d5212-5555-4f0a-9cf5-98b0a9e9d8db None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:36.519242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b999870-0b62-4568-8e96-ef78bee7c956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:36.520794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999870-0b62-4568-8e96-ef78bee7c956 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.520794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999870-0b62-4568-8e96-ef78bee7c956 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.520794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b999870-0b62-4568-8e96-ef78bee7c956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.549853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 803/1610] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.551108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.551402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.551676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.558442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:36.575388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.576630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.638537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b999870-0b62-4568-8e96-ef78bee7c956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:36.640028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b999870-0b62-4568-8e96-ef78bee7c956 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.652154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 808/1611] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:36.655657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2c6f-4d45-416b-ae79-6094aa4ec0ae None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.655996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2c6f-4d45-416b-ae79-6094aa4ec0ae None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.656275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2c6f-4d45-416b-ae79-6094aa4ec0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.670585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 804/1612] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.672468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.682764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b73c2c6f-4d45-416b-ae79-6094aa4ec0ae None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.683796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b73c2c6f-4d45-416b-ae79-6094aa4ec0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.694531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e5f99-b471-449f-ac2a-fa31235d610a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:36.700544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 809/1613] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.700544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.700544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.700544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.703605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b26401-572c-4279-9121-f259d03af10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:36.704335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b26401-572c-4279-9121-f259d03af10d service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.704533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b26401-572c-4279-9121-f259d03af10d service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.704809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b26401-572c-4279-9121-f259d03af10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.723489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b26401-572c-4279-9121-f259d03af10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:36.725009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b26401-572c-4279-9121-f259d03af10d service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.743476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 805/1614] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:36.744929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.745225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.747885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.749491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] MFA Rules not processed for user `ebdc4f869011444b816472914ab73dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.751153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.853240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:36.854905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:36.865193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 810/1615] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.868038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.893718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac915f3-8f97-4247-ab7f-af6877b167b7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:36.905572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:36.906398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:36.906817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:36.907276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:36.908490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:36.912557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:36.936384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:36.948646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 811/1616] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:36.949929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:36.990060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 806/1617] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Jul 20 21:52:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:36.991441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.004115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7527df-ba66-486a-a04d-5d7ae24b1a14 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.041646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b423a03-0faa-48ca-bcad-548333a504d9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.047436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.048359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.048895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.049252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.083806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.085309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.085440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.086016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.112192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:37.173824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.175575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.175713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.178813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.198861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 812/1618] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:37.200719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.208057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 807/1619] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:37.209062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.215623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3220e2-a44e-4bae-921e-392a1bb5601d service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.221009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.221916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.222497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.222778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.223686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.226034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.226194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2ef30-902e-4ddd-8000-7b4d22b0a77c service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.245128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.245837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.246124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.246405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.249122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.249575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.266280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 813/1620] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:37 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:37.267645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.294220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 808/1621] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:37 2022] POST /identity/v3/users => generated 289 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:37.294499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.323161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a7e1-0899-4b3a-a499-0b93a6aa6591 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.402988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5ffe1-d57b-4afc-b36e-a5d59ae21ec8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.414998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.418241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.418628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.419190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.446229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.446872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.447202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.447484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.448771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.451496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.467116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:37.485949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 809/1622] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:37 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:37.489398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.506642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5e625-b8f5-49db-87f9-a6b6ff18d01b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.514149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.515568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.516346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.516703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.518078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.520114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.535832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.539943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.546217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 810/1623] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:37.547691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.555934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 814/1624] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:37.556974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.574840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3a223-1c9f-4dd9-98fa-68172b1a37ad admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.578091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1022839-cd3c-49f9-9a89-94f3e732f62f service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.585329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.586465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.586745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.587106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] PATH_INFO: `/v3/projects/d4534d4792d244d781a13a79fb2da860/users/e05359ca9c4c47d2a7682790493289b4/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.587217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.587952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.588234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.588512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.589678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.592592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.617057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4534d4792d244d781a13a79fb2da860, user_id=e05359ca9c4c47d2a7682790493289b4, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.622678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.625510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 811/1625] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:37.627828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.642941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 815/1626] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:37 2022] PUT /identity/v3/projects/d4534d4792d244d781a13a79fb2da860/users/e05359ca9c4c47d2a7682790493289b4/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:37.650896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.653560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ebc89-c101-4524-8f18-6a1dee92aaec admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.700069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:37.773018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63431bf3-a1c0-4f9c-a6d4-b463e2ae66f9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.778752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.779510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.779797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.780079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.780832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.782887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.812466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.816802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.816802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.816802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] PATH_INFO: `/v3/projects/f788304fcb464688b7f0930c501ed422/users/eacb46e37d7d42ec9f7caceee72bcb08/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.818142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 816/1627] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:37.819388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.846488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91294d1e-85e1-4c0a-b5f6-b59793cf43cb admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:37.858529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f788304fcb464688b7f0930c501ed422, user_id=eacb46e37d7d42ec9f7caceee72bcb08, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.863522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.866795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18402b43-5e5b-4d84-b1d0-6c4ee9927d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:37.868100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402b43-5e5b-4d84-b1d0-6c4ee9927d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.869093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402b43-5e5b-4d84-b1d0-6c4ee9927d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.869569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18402b43-5e5b-4d84-b1d0-6c4ee9927d5c admin admin] PATH_INFO: `/v3/projects/d4534d4792d244d781a13a79fb2da860/users/e05359ca9c4c47d2a7682790493289b4/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.872931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18402b43-5e5b-4d84-b1d0-6c4ee9927d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4534d4792d244d781a13a79fb2da860, user_id=e05359ca9c4c47d2a7682790493289b4, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:37.918997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18402b43-5e5b-4d84-b1d0-6c4ee9927d5c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:37.942104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 812/1628] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:37 2022] PUT /identity/v3/projects/f788304fcb464688b7f0930c501ed422/users/eacb46e37d7d42ec9f7caceee72bcb08/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:37.942104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:37.942618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 817/1629] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:37 2022] PUT /identity/v3/projects/d4534d4792d244d781a13a79fb2da860/users/e05359ca9c4c47d2a7682790493289b4/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:37.942618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:37.942618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:37.942618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:37.974209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] MFA Rules not processed for user `e05359ca9c4c47d2a7682790493289b4`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:37.975797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:38.059853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d0bde-26f7-48fc-bc8f-f43f26138ae4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:38.105641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 818/1630] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:38.133746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:38.210475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8dfd4-8ccd-43a6-ae27-accef477a3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:38.211769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8dfd4-8ccd-43a6-ae27-accef477a3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.212158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8dfd4-8ccd-43a6-ae27-accef477a3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.212530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8dfd4-8ccd-43a6-ae27-accef477a3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.214345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8dfd4-8ccd-43a6-ae27-accef477a3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:38.216650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d8dfd4-8ccd-43a6-ae27-accef477a3b4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:38.226860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:38.244012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47097cf-5beb-461c-a54d-76d14744805c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:38.245886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 813/1631] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:38.253835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79b7d-8da7-423d-8e7d-91de7f71cd1b None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.254126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79b7d-8da7-423d-8e7d-91de7f71cd1b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.254407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79b7d-8da7-423d-8e7d-91de7f71cd1b None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.254538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec915f-e63c-4f58-a088-6d6c67816347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:38.255416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec915f-e63c-4f58-a088-6d6c67816347 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.255842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec915f-e63c-4f58-a088-6d6c67816347 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.256551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 814/1632] 10.222.0.86 () {54 vars in 912 bytes} [Wed Jul 20 21:52:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:52:38.256686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec915f-e63c-4f58-a088-6d6c67816347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.257012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb79b7d-8da7-423d-8e7d-91de7f71cd1b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:38.273439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec915f-e63c-4f58-a088-6d6c67816347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:38.275828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec915f-e63c-4f58-a088-6d6c67816347 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:38.286727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb79b7d-8da7-423d-8e7d-91de7f71cd1b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:38.304962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ac1c3-a24e-4e94-822f-06f99ed4fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:38.305278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 819/1633] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:38.305870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac1c3-a24e-4e94-822f-06f99ed4fa5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.306162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac1c3-a24e-4e94-822f-06f99ed4fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.306464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ac1c3-a24e-4e94-822f-06f99ed4fa5a admin admin] PATH_INFO: `/v3/projects/f788304fcb464688b7f0930c501ed422/users/eacb46e37d7d42ec9f7caceee72bcb08/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.307519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.307917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.308207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.308604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ac1c3-a24e-4e94-822f-06f99ed4fa5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f788304fcb464688b7f0930c501ed422, user_id=eacb46e37d7d42ec9f7caceee72bcb08, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:38.311283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ac1c3-a24e-4e94-822f-06f99ed4fa5a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:38.334017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:38.336226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:38.348869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 815/1634] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:38 2022] PUT /identity/v3/projects/f788304fcb464688b7f0930c501ed422/users/eacb46e37d7d42ec9f7caceee72bcb08/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:38.354667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.355240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.355696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.393639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:38.427443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] MFA Rules not processed for user `eacb46e37d7d42ec9f7caceee72bcb08`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:38.429812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:38.510623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 820/1635] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:52:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:38.515880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:38.533263 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 816/1636] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:38.540076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:38.674874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc0186-6bd8-40ab-85dc-ecc15eb58bf2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:38.685581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:38.686418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.686777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.687174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.693250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c8af-c647-4b23-af83-1f632b84de4d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:38.708555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:38.709847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:38.710169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:38.710513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:38.725924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:38.727546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:38.744920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 817/1637] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:38.822307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:38.825462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:38.842598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 821/1638] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:39.571504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:39.580726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ee9d8-4829-4935-a0f1-f0d56d305a99 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:39.585059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742aea6-eccc-4d5b-9e5d-f7bb1103c795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:39.585822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742aea6-eccc-4d5b-9e5d-f7bb1103c795 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:39.586111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742aea6-eccc-4d5b-9e5d-f7bb1103c795 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:39.586393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742aea6-eccc-4d5b-9e5d-f7bb1103c795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:39.601917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742aea6-eccc-4d5b-9e5d-f7bb1103c795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:39.603503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742aea6-eccc-4d5b-9e5d-f7bb1103c795 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:39.629214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 818/1639] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:39.737003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:39.751116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed961c88-6244-4385-a210-8a19b0b21e2d service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:39.763815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bfaab-edc0-4a5e-9222-24e0325e1a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:39.764506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bfaab-edc0-4a5e-9222-24e0325e1a20 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:39.764786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bfaab-edc0-4a5e-9222-24e0325e1a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:39.765072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bfaab-edc0-4a5e-9222-24e0325e1a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:39.910067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bfaab-edc0-4a5e-9222-24e0325e1a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:39.919150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bfaab-edc0-4a5e-9222-24e0325e1a20 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:39.949011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 822/1640] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:40.458014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96131850-9756-4ad0-8780-1394fc2052a8 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:40.458864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96131850-9756-4ad0-8780-1394fc2052a8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:40.458864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96131850-9756-4ad0-8780-1394fc2052a8 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:40.461587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 819/1641] 10.222.0.86 () {54 vars in 962 bytes} [Wed Jul 20 21:52:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:52:40.474722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7979001-54ca-4d43-abdc-e9aa11bcaa22 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:40.475598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7979001-54ca-4d43-abdc-e9aa11bcaa22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:40.475900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7979001-54ca-4d43-abdc-e9aa11bcaa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:40.527024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a7979001-54ca-4d43-abdc-e9aa11bcaa22 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:40.534128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a7979001-54ca-4d43-abdc-e9aa11bcaa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:40.562151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 823/1642] 10.222.0.86 () {58 vars in 1059 bytes} [Wed Jul 20 21:52:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:40.571267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80e55b-c034-4d59-b640-f54c84073cd1 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:40.571621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80e55b-c034-4d59-b640-f54c84073cd1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:40.572404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80e55b-c034-4d59-b640-f54c84073cd1 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:40.573796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 820/1643] 10.222.0.86 () {54 vars in 912 bytes} [Wed Jul 20 21:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:52:40.583483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:40.584029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:40.584922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:40.626244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:40.627681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:40.649172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 824/1644] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:52:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:40.679235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80e55b-c034-4d59-b640-f54c84073cd1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:40.695327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80e55b-c034-4d59-b640-f54c84073cd1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:40.700471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b6c1b-a778-4e47-b762-7722f463292e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:40.701546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b6c1b-a778-4e47-b762-7722f463292e service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:40.702395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b6c1b-a778-4e47-b762-7722f463292e service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:40.702895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b6c1b-a778-4e47-b762-7722f463292e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:40.739447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b6c1b-a778-4e47-b762-7722f463292e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:40.743755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b6c1b-a778-4e47-b762-7722f463292e service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:40.794874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 821/1645] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:44.443688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:44.461653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:44.620993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0e918-16a1-44de-8616-9fadde8ee29d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:44.627460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:44.628565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:44.630316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:44.630717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] PATH_INFO: `/v3/users/65eed315b2f249e79a6a27b97a55102d` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:44.636191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] RBAC: Authorizing `identity:delete_user(user_id=65eed315b2f249e79a6a27b97a55102d)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:44.642382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:44.786252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65eed315b2f249e79a6a27b97a55102d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:44.806678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65eed315b2f249e79a6a27b97a55102d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:44.817249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65eed315b2f249e79a6a27b97a55102d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:44.832518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65eed315b2f249e79a6a27b97a55102d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:44.833410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65eed315b2f249e79a6a27b97a55102d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:44.862815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f151-3b36-4cfc-9ea8-ce9e9367f161 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:44.864177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f151-3b36-4cfc-9ea8-ce9e9367f161 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:44.864735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f151-3b36-4cfc-9ea8-ce9e9367f161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:44.869259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 825/1646] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:44 2022] DELETE /identity/v3/users/65eed315b2f249e79a6a27b97a55102d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:44.931047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0fd6f151-3b36-4cfc-9ea8-ce9e9367f161 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:44.932983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0fd6f151-3b36-4cfc-9ea8-ce9e9367f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:45.111148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:45.195801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 822/1647] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:45.337472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:45.340420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:45.591317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871fdae-4ecf-4f6e-ab98-951b3d1e03df admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:45.603793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:45.604741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:45.605029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:45.605308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:45.628427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:45.629737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:45.659324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 826/1648] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:52:46.573123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:46.573659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:46.573744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:46.685882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] MFA Rules not processed for user `c58389cb50e140d69f1d4c99d6d0e182`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:46.691454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:46.961436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 823/1649] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:46.971247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:46.972861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:47.141075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:47.287128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabce1e6-55ba-4974-974c-35f52eee0208 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:47.307525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:47.310833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:47.323588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:47.325200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:47.341293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:47.343977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:47.358177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 827/1650] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:47.978963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:47.980862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:48.013647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bb8f1-6681-4a0a-958a-9cc911e9eb34 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:48.032098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:48.033006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:48.033291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:48.033569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] PATH_INFO: `/v3/projects/93b13ad4193f4aecb19e375a1cb82516` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:48.036639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b13ad4193f4aecb19e375a1cb82516)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:48.039338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:48.215624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b13ad4193f4aecb19e375a1cb82516', 'request_id': 'req-c68fef12-a758-4a09-89bf-b156be59079b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.230374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b13ad4193f4aecb19e375a1cb82516', 'request_id': 'req-c68fef12-a758-4a09-89bf-b156be59079b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.255953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b13ad4193f4aecb19e375a1cb82516', 'request_id': 'req-c68fef12-a758-4a09-89bf-b156be59079b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.282088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] The token cache is being invalidate because project 93b13ad4193f4aecb19e375a1cb82516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:52:48.284757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.293181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.308333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:48.308923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 824/1651] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:52:47 2022] DELETE /identity/v3/projects/93b13ad4193f4aecb19e375a1cb82516 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:48.325876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518fede-ae35-45b7-b2cd-0293cd5a7c8d service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:48.407327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:48.408871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:48.535648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:48.536502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:48.536781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:48.543454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] PATH_INFO: `/v3/users/e05359ca9c4c47d2a7682790493289b4` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:48.548201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05359ca9c4c47d2a7682790493289b4)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:48.552354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:48.596735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fef12-a758-4a09-89bf-b156be59079b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:48.616557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:48.618357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:48.619536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:48.619988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] PATH_INFO: `/v3/users/69a5796d5cbf4c6a9a8de1c9aa9c6ccf` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:48.625268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a5796d5cbf4c6a9a8de1c9aa9c6ccf)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:48.628856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: WARNING py.warnings [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:52:48.628856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: warnings.warn(msg) Jul 20 21:52:48.628856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: Jul 20 21:52:48.631964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:48.642455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:48.682712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05359ca9c4c47d2a7682790493289b4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.719857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05359ca9c4c47d2a7682790493289b4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.741660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05359ca9c4c47d2a7682790493289b4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.764759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05359ca9c4c47d2a7682790493289b4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.768943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05359ca9c4c47d2a7682790493289b4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.810945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 828/1652] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:48 2022] DELETE /identity/v3/users/e05359ca9c4c47d2a7682790493289b4 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:48.922733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a5796d5cbf4c6a9a8de1c9aa9c6ccf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:48.971102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:48.972473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:48.986473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a5796d5cbf4c6a9a8de1c9aa9c6ccf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.041393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a5796d5cbf4c6a9a8de1c9aa9c6ccf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.043834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a5796d5cbf4c6a9a8de1c9aa9c6ccf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.047701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-010845f8-20fe-49c1-a4fb-126c75862a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a5796d5cbf4c6a9a8de1c9aa9c6ccf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.083841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 825/1653] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:48 2022] DELETE /identity/v3/users/69a5796d5cbf4c6a9a8de1c9aa9c6ccf => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:49.091314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:49.091731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:49.092988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:49.093743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:49.147460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:49.149021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:49.180835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ebc4-2a71-43d9-84b7-bfa74c48d72f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:49.352485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 826/1654] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:49.367449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:49.368796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:49.369057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:49.369328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] PATH_INFO: `/v3/users/ebdc4f869011444b816472914ab73dd7` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:49.371867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebdc4f869011444b816472914ab73dd7)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:49.373855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:49.398848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:49.400849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:49.451706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebdc4f869011444b816472914ab73dd7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.474366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebdc4f869011444b816472914ab73dd7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.485452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebdc4f869011444b816472914ab73dd7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.507855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebdc4f869011444b816472914ab73dd7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.508367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-aee35aa1-340b-41a6-bfd6-c5a21c56537e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebdc4f869011444b816472914ab73dd7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:49.517119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 829/1655] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:48 2022] DELETE /identity/v3/users/ebdc4f869011444b816472914ab73dd7 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:52:49.522924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:49.522924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:49.523289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:49.570294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:49.570496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2dae5-ad01-41d5-8738-8f905714e5bc None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:49.571575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:49.644061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:49.676699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 830/1656] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:49.679003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:49.716672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:49.717497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:49.717857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:49.718137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:49.732721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:49.808615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:49.810377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:49.866143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 827/1657] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:52:49.987876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:49.989609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:50.017893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e86cc0c-36cb-4d79-a12a-787f77a1c953 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:50.033321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f25aa3-a482-4438-9b9b-da153c1e9213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:50.034426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25aa3-a482-4438-9b9b-da153c1e9213 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:50.035447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25aa3-a482-4438-9b9b-da153c1e9213 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:50.035447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f25aa3-a482-4438-9b9b-da153c1e9213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:50.087814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25aa3-a482-4438-9b9b-da153c1e9213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:50.094624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f25aa3-a482-4438-9b9b-da153c1e9213 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:50.130043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 831/1658] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:50.239938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:50.242416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:50.266526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7e24e-ebef-420f-ac74-ef94ef39250e service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:50.284460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:50.285724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:50.286145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:50.288006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] PATH_INFO: `/v3/projects/d4534d4792d244d781a13a79fb2da860` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:50.290465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4534d4792d244d781a13a79fb2da860)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:50.296299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:50.411552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4534d4792d244d781a13a79fb2da860', 'request_id': 'req-b33036c8-d5d5-4279-a5f3-8ab045f4859a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:50.429589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4534d4792d244d781a13a79fb2da860', 'request_id': 'req-b33036c8-d5d5-4279-a5f3-8ab045f4859a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:50.446597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4534d4792d244d781a13a79fb2da860', 'request_id': 'req-b33036c8-d5d5-4279-a5f3-8ab045f4859a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:50.465983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] The token cache is being invalidate because project d4534d4792d244d781a13a79fb2da860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:52:50.466416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:50.468431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b33036c8-d5d5-4279-a5f3-8ab045f4859a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:50.476682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 828/1659] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:52:50 2022] DELETE /identity/v3/projects/d4534d4792d244d781a13a79fb2da860 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:50.478130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:50.478604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:50.478691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:50.528058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:50.536895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:50.774514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 832/1660] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:50.775928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:50.776296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:50.776672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:50.826555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:50.832972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:50.854962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 829/1661] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:50.861197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:50.881955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbf040-a0d4-4a08-a9e5-1e4d42f56be7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:50.897343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:50.898043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:50.898492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:50.898763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:50.899599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:50.899853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:50.901638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:50.932865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:50.933049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 833/1662] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:50.952052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854fbf-793d-4076-80c9-209f2ec26379 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:50.966827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:50.967691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:50.967970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:50.968244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:50.968958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:50.971010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:51.032132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 830/1663] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:50 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:52:51.035993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:51.051517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba2fd7-7514-4947-afc2-3825efa8b680 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:51.198167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:51.198900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:51.199161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:51.199435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:51.200300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:51.202220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:51.251801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 834/1664] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:51 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:52:51.256803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:51.273207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd0a76-ae0b-4b57-88e3-5648c6ea0a16 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:51.285494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:51.286556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:51.286976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:51.287512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:51.288572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:51.292745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:51.323356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:51.323925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 831/1665] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:51.340685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758082bf-ddab-4d5e-8281-c22cdc283f75 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:51.353150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d213bf-9ebf-4c9f-8b13-f44cb02cd6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:51.353852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d213bf-9ebf-4c9f-8b13-f44cb02cd6bc admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:51.354296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d213bf-9ebf-4c9f-8b13-f44cb02cd6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:51.354751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d213bf-9ebf-4c9f-8b13-f44cb02cd6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:51.356018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d213bf-9ebf-4c9f-8b13-f44cb02cd6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:51.358045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d213bf-9ebf-4c9f-8b13-f44cb02cd6bc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:51.375073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 835/1666] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:51.379096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:51.398726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6fc77-f47e-4e96-a04e-3f47d85decba admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:51.406712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:51.408350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:51.408350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:51.408350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] PATH_INFO: `/v3/projects/eb4e21826f7e4a32bde9ec4e8db0565d/users/674572f6b113465591a0603c2d130ca7/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:51.435676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4e21826f7e4a32bde9ec4e8db0565d, user_id=674572f6b113465591a0603c2d130ca7, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:51.443543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:51.482800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 832/1667] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:51 2022] PUT /identity/v3/projects/eb4e21826f7e4a32bde9ec4e8db0565d/users/674572f6b113465591a0603c2d130ca7/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:51.495675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:51.496172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:51.496248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:51.577672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] MFA Rules not processed for user `674572f6b113465591a0603c2d130ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:51.579027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:51.721742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:51.785878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 836/1668] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:51.792572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.009321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:52.024143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9e2e-650c-4d6b-919b-22fca1c81ba3 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.033850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.034876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.035359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.035818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.036705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.045833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:52.101526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 833/1669] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:51 2022] POST /identity/v3/projects => generated 387 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:52.105465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.130497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5cae1-4fda-4d16-88d1-1b895f146d50 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.219391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.220430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.220927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.221466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.222484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.225913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:52.258039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 837/1670] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:52 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:52:52.261722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.282870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c5d26-3c55-4796-b9c7-01f6f0fd2a5e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.293324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.294507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.295373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.296164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.297317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.300537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:52.316346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 834/1671] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:52:52.320338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.337627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d4509-ce88-4521-9668-773beb015cd5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.351959 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.354117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.354516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.354967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] PATH_INFO: `/v3/projects/9717fbd451374ef08e0779487f95b714/users/7ce08745c27f4bbba1b8830740e761c1/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.380295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9717fbd451374ef08e0779487f95b714, user_id=7ce08745c27f4bbba1b8830740e761c1, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.385238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:52.406405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 838/1672] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:52 2022] PUT /identity/v3/projects/9717fbd451374ef08e0779487f95b714/users/7ce08745c27f4bbba1b8830740e761c1/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:52.410313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.428987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966db74-3906-48c7-9192-e0c4ba4056fe admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.528828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8222d-d766-43e6-8ff7-5f2c9f37c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.529862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8222d-d766-43e6-8ff7-5f2c9f37c9bf admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.530615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8222d-d766-43e6-8ff7-5f2c9f37c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.531251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8222d-d766-43e6-8ff7-5f2c9f37c9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.532334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8222d-d766-43e6-8ff7-5f2c9f37c9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.535712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8222d-d766-43e6-8ff7-5f2c9f37c9bf admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:52.554795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 835/1673] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:52.559010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.573333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156afc6-e992-4d43-8e7c-aba3ae6b26e3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.579525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.580916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.581389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.581852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] PATH_INFO: `/v3/projects/9717fbd451374ef08e0779487f95b714/users/7ce08745c27f4bbba1b8830740e761c1/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.586692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9717fbd451374ef08e0779487f95b714, user_id=7ce08745c27f4bbba1b8830740e761c1, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.590327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:52.613461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 839/1674] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:52 2022] PUT /identity/v3/projects/9717fbd451374ef08e0779487f95b714/users/7ce08745c27f4bbba1b8830740e761c1/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:52:52.624974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718e540-3102-4921-95c4-491477ea96ec None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.625291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718e540-3102-4921-95c4-491477ea96ec None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.625529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718e540-3102-4921-95c4-491477ea96ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.664561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-f718e540-3102-4921-95c4-491477ea96ec None None] MFA Rules not processed for user `7ce08745c27f4bbba1b8830740e761c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:52.665859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f718e540-3102-4921-95c4-491477ea96ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:52.668965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.671080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:52.928959 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbc584-d17b-4456-87ac-48684d6ce524 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:52.929765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 836/1675] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:52.946684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:52.948119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718e540-3102-4921-95c4-491477ea96ec None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:52.949267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:52.950079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:52.950192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f718e540-3102-4921-95c4-491477ea96ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:52.950303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] PATH_INFO: `/v3/projects/99ddb19705984622843d5020633d6c31` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:52.954657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ddb19705984622843d5020633d6c31)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:52.955926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: WARNING py.warnings [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 21:52:52.955926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: warnings.warn(msg) Jul 20 21:52:52.955926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: Jul 20 21:52:52.957380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:53.065087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:53.098863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ddb19705984622843d5020633d6c31', 'request_id': 'req-e039a2b6-deac-496d-a293-413141dad242'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.112135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718e540-3102-4921-95c4-491477ea96ec None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:53.122695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:53.148471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ddb19705984622843d5020633d6c31', 'request_id': 'req-e039a2b6-deac-496d-a293-413141dad242'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.172761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ddb19705984622843d5020633d6c31', 'request_id': 'req-e039a2b6-deac-496d-a293-413141dad242'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.197129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] The token cache is being invalidate because project 99ddb19705984622843d5020633d6c31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:52:53.197481 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.199430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.201275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 840/1676] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:52:52 2022] DELETE /identity/v3/projects/99ddb19705984622843d5020633d6c31 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:53.205885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:53.207461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:53.207593 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:53.208135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:53.208415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:53.209905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:53.212539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:53.353370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:53.354918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:53.369273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 837/1677] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:53.371263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a2b6-deac-496d-a293-413141dad242 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:53.385817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:53.387094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:53.387387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:53.387665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] PATH_INFO: `/v3/users/eacb46e37d7d42ec9f7caceee72bcb08` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:53.389589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eacb46e37d7d42ec9f7caceee72bcb08)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:53.391606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:53.536102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eacb46e37d7d42ec9f7caceee72bcb08'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.570655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eacb46e37d7d42ec9f7caceee72bcb08'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.580331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eacb46e37d7d42ec9f7caceee72bcb08'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.592554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eacb46e37d7d42ec9f7caceee72bcb08'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.593636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eacb46e37d7d42ec9f7caceee72bcb08'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:53.603821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 841/1678] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:53 2022] DELETE /identity/v3/users/eacb46e37d7d42ec9f7caceee72bcb08 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:53.640596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:53.642240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:53.664407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de649935-7364-4780-88e2-a57365b70a12 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:53.821549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:53.822974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:53.824018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:53.824937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] PATH_INFO: `/v3/users/674572f6b113465591a0603c2d130ca7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:53.834635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674572f6b113465591a0603c2d130ca7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:53.844688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:53.860496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:53.862179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:53.923292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb98bfc-d7c5-4cec-9900-f79531f3e125 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:53.937074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:53.938192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:53.938489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:53.938771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] PATH_INFO: `/v3/projects/8a1f1bb699504cb38e176e5dabf3040e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:53.949772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a1f1bb699504cb38e176e5dabf3040e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:53.952586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:54.027921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674572f6b113465591a0603c2d130ca7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.064116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:54.065342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:54.069360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674572f6b113465591a0603c2d130ca7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.099565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674572f6b113465591a0603c2d130ca7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.101455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674572f6b113465591a0603c2d130ca7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.102276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6a44d7a7-32ca-4a53-a6b7-e74b77306e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674572f6b113465591a0603c2d130ca7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.105247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a1f1bb699504cb38e176e5dabf3040e', 'request_id': 'req-901f5f74-904a-4c91-92cb-f2a305a28a79'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.124016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a1f1bb699504cb38e176e5dabf3040e', 'request_id': 'req-901f5f74-904a-4c91-92cb-f2a305a28a79'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.136448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 838/1679] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:53 2022] DELETE /identity/v3/users/674572f6b113465591a0603c2d130ca7 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:54.138452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.138857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.139143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.141295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a1f1bb699504cb38e176e5dabf3040e', 'request_id': 'req-901f5f74-904a-4c91-92cb-f2a305a28a79'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.170705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] The token cache is being invalidate because project 8a1f1bb699504cb38e176e5dabf3040e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:52:54.171039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.172408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-901f5f74-904a-4c91-92cb-f2a305a28a79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:54.179757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 842/1680] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:52:53 2022] DELETE /identity/v3/projects/8a1f1bb699504cb38e176e5dabf3040e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:52:54.183309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e39a6-cc69-4606-8a58-5bb623e3d2bc None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.183598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e39a6-cc69-4606-8a58-5bb623e3d2bc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.184070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e39a6-cc69-4606-8a58-5bb623e3d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.194886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:54.197005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:54.242223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d07e39a6-cc69-4606-8a58-5bb623e3d2bc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:54.243801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d07e39a6-cc69-4606-8a58-5bb623e3d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:54.299560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 839/1681] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:54.396173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 843/1682] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:54.397873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:54.399357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:54.405966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.409969 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.417463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.475486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:54.478976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:54.500617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 844/1683] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:54.502063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:54.522076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb5557-1f17-47a9-b600-ac8e25ab2967 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:54.528488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:54.529237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.529453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.529734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.531077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:54.531442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:54.533503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:54.554780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 845/1684] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:54.562566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:54.571960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc496862-f4a4-4660-b1cd-ad9fd8a507c1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:54.590399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866698d-4b82-44d5-a223-33c4586a21bb admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:54.594487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:54.595783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.596258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.596547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.604705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:54.604897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.605175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.605461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.606147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:54.608168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:54.622076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:54.623839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:54.652445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 840/1685] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:54.712790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 846/1686] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:54 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:54.714031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:54.728053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b366c-a752-4373-856c-6dec5c655095 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:54.905718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:54.906626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:54.907012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:54.907426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:54.908460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:54.918234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:54.971227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 841/1687] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:54 2022] POST /identity/v3/users => generated 287 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:52:54.973741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:54.993421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb615b6-ebf1-42d8-abd5-e6e0000e4b37 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:55.011500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:55.012236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.013024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.013024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.013535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:55.016922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:55.049236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 847/1688] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:55.050627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:55.066852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6099e-214f-4fef-8327-dcf43b57f7b7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:55.092692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d068f-f281-45f7-809d-419dac726ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:55.093387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d068f-f281-45f7-809d-419dac726ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.093673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d068f-f281-45f7-809d-419dac726ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.093954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d068f-f281-45f7-809d-419dac726ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.094703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d068f-f281-45f7-809d-419dac726ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:55.102743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d068f-f281-45f7-809d-419dac726ef8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:55.116278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:55.139988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 842/1689] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:55.141283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:55.158604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404eb1d7-81d2-44ca-8197-481258a0e5ef admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:55.168059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6523caf-71d4-4ebc-b06d-bc95be7611ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:55.168986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6523caf-71d4-4ebc-b06d-bc95be7611ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.169270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6523caf-71d4-4ebc-b06d-bc95be7611ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.169549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6523caf-71d4-4ebc-b06d-bc95be7611ee admin admin] PATH_INFO: `/v3/projects/948ccdc261db41feb6d0b34bc8c9e1a6/users/d93fa1bf91d54079b3e0a73436d749ce/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.219845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943db4d-0c5a-4fb0-bc65-22635d08c627 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.220190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943db4d-0c5a-4fb0-bc65-22635d08c627 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.220408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943db4d-0c5a-4fb0-bc65-22635d08c627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.259920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6523caf-71d4-4ebc-b06d-bc95be7611ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=948ccdc261db41feb6d0b34bc8c9e1a6, user_id=d93fa1bf91d54079b3e0a73436d749ce, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:55.267920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6523caf-71d4-4ebc-b06d-bc95be7611ee admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:55.316978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4943db4d-0c5a-4fb0-bc65-22635d08c627 None None] MFA Rules not processed for user `74e755a5fe9f4c5c95376beae691381f`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:55.318804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4943db4d-0c5a-4fb0-bc65-22635d08c627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:55.324975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 848/1690] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:55 2022] PUT /identity/v3/projects/948ccdc261db41feb6d0b34bc8c9e1a6/users/d93fa1bf91d54079b3e0a73436d749ce/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:55.336149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.336364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.336642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.402606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] MFA Rules not processed for user `d93fa1bf91d54079b3e0a73436d749ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:55.415445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:55.541139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 843/1691] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:55.551895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.552614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.553074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.554924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:55.577765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] MFA Rules not processed for user `74e755a5fe9f4c5c95376beae691381f`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:55.579477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:55.602449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 844/1692] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:55.613337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:55.614958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:55.631827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 849/1693] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:55.648163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:55.649616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:55.802104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9923f-23ea-4043-bfb3-2a8ed637a969 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:55.810594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:55.811607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.811996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.812378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.814758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff6d90-3f49-4d4e-93e3-4b935dbb5b73 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:55.825210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:55.830164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:55.830377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:55.830870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:55.831183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:55.831468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:55.844881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:55.846522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:55.848963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 845/1694] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:52:55.906545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 850/1695] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:56.172873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:56.174353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:56.434211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58634c4c-c04f-44da-91b3-7ca8ccd90ff5 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:56.440486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:56.441380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:56.441380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:56.441656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] PATH_INFO: `/v3/projects/f788304fcb464688b7f0930c501ed422` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:56.453021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f788304fcb464688b7f0930c501ed422)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:56.455997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:56.490693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:56.582864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:56.584533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:56.602009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f788304fcb464688b7f0930c501ed422', 'request_id': 'req-a5dc322e-969b-41a7-a1d2-500ae0c11f11'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:56.625466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14345cc0-1108-455f-b121-e12cefef12af service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:56.626762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f788304fcb464688b7f0930c501ed422', 'request_id': 'req-a5dc322e-969b-41a7-a1d2-500ae0c11f11'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:56.681358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f788304fcb464688b7f0930c501ed422', 'request_id': 'req-a5dc322e-969b-41a7-a1d2-500ae0c11f11'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:56.739838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] The token cache is being invalidate because project f788304fcb464688b7f0930c501ed422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:52:56.756435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:56.759618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a5dc322e-969b-41a7-a1d2-500ae0c11f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:56.764501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 846/1696] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:52:56 2022] DELETE /identity/v3/projects/f788304fcb464688b7f0930c501ed422 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:56.778587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276e344-ac8c-4747-b7c4-39431b8507b6 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:56.779514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276e344-ac8c-4747-b7c4-39431b8507b6 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:56.781579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276e344-ac8c-4747-b7c4-39431b8507b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:56.841798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9276e344-ac8c-4747-b7c4-39431b8507b6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:56.847996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9276e344-ac8c-4747-b7c4-39431b8507b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:56.893833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:56.894726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:56.895018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:56.896564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] PATH_INFO: `/v3/projects/eb4e21826f7e4a32bde9ec4e8db0565d` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:56.899845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4e21826f7e4a32bde9ec4e8db0565d)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:56.907693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:57.027335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 847/1697] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:57.030111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdd18c-d1ce-4a97-abc9-cfa18953fcef None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.030881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdd18c-d1ce-4a97-abc9-cfa18953fcef None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.031438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdd18c-d1ce-4a97-abc9-cfa18953fcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:57.064642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-6dfdd18c-d1ce-4a97-abc9-cfa18953fcef None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:57.064642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6dfdd18c-d1ce-4a97-abc9-cfa18953fcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:57.098949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4e21826f7e4a32bde9ec4e8db0565d', 'request_id': 'req-85af8359-f8f8-4ce8-9063-bf6285dcdee2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.119638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4e21826f7e4a32bde9ec4e8db0565d', 'request_id': 'req-85af8359-f8f8-4ce8-9063-bf6285dcdee2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.136033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4e21826f7e4a32bde9ec4e8db0565d', 'request_id': 'req-85af8359-f8f8-4ce8-9063-bf6285dcdee2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.151085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:57.182864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] The token cache is being invalidate because project eb4e21826f7e4a32bde9ec4e8db0565d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:52:57.186098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.189459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.201663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 851/1698] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:52:56 2022] DELETE /identity/v3/projects/eb4e21826f7e4a32bde9ec4e8db0565d => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:52:57.206903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:57.212841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:57.306540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 848/1699] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:57.309258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.312182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.312572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:57.422633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] MFA Rules not processed for user `c58389cb50e140d69f1d4c99d6d0e182`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:57.424639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:57.559638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:57.606983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af8359-f8f8-4ce8-9063-bf6285dcdee2 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:57.629851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:57.633017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.633883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.634667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] PATH_INFO: `/v3/users/7ce08745c27f4bbba1b8830740e761c1` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:57.637523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce08745c27f4bbba1b8830740e761c1)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:57.645777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:57.659132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 849/1700] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:57.660591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:57.708099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78695793-0784-4ca2-9bda-2b257c1aede8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:57.718747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:57.719819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.720426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.720880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:57.722142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:52:57.722631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:57.730900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55cd2a-6ae2-49f2-a907-eb2eec52617c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:57.778922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 850/1701] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:52:57.783580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c07c-2868-4e1f-8619-534c015d18e7 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.785851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c07c-2868-4e1f-8619-534c015d18e7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.786369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c07c-2868-4e1f-8619-534c015d18e7 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:57.796096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 851/1702] 10.222.0.86 () {54 vars in 912 bytes} [Wed Jul 20 21:52:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:52:57.798521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6c07c-2868-4e1f-8619-534c015d18e7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:57.809420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce08745c27f4bbba1b8830740e761c1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.849290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce08745c27f4bbba1b8830740e761c1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.876337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce08745c27f4bbba1b8830740e761c1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.876580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6c07c-2868-4e1f-8619-534c015d18e7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:57.885378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf969c-4a03-4198-8b56-7433819e9305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:57.886251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf969c-4a03-4198-8b56-7433819e9305 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.886626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf969c-4a03-4198-8b56-7433819e9305 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.886981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf969c-4a03-4198-8b56-7433819e9305 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:57.888810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf969c-4a03-4198-8b56-7433819e9305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:57.889613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce08745c27f4bbba1b8830740e761c1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.890591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d42105c5-7772-458d-82f7-ede20b46b0e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce08745c27f4bbba1b8830740e761c1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:52:57.896779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf969c-4a03-4198-8b56-7433819e9305 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:57.912061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 852/1703] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:52:57 2022] DELETE /identity/v3/users/7ce08745c27f4bbba1b8830740e761c1 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:57.947299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8171d4-5bdf-4255-b1ee-05e7d525a20d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:57.947299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8171d4-5bdf-4255-b1ee-05e7d525a20d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:57.947299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8171d4-5bdf-4255-b1ee-05e7d525a20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.007381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 852/1704] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:57 2022] POST /identity/v3/projects => generated 405 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:58.007381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.007381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.007381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.068937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0d8171d4-5bdf-4255-b1ee-05e7d525a20d None None] MFA Rules not processed for user `7f38c9f8b5ea438dbae61a15c1e84e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:58.077238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0d8171d4-5bdf-4255-b1ee-05e7d525a20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:58.092570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] MFA Rules not processed for user `1a0d4ce0ac4d47c8a184ea0291461379`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:58.093972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:58.202676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 853/1705] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:58.204307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:58.206246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:58.274661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 853/1706] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:58.278965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.279292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.279577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.322623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] MFA Rules not processed for user `1a0d4ce0ac4d47c8a184ea0291461379`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:58.325036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:58.344401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 854/1707] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:58.345269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:58.347037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:58.366234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:58.368951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8cb48-1d4b-4fb1-a676-307ba31010cc None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:58.378148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7927c4-440e-4d33-875c-ddc521cd4c4a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:58.379140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:58.379890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.380174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.380451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.381267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:58.385677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:58.388750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:58.389419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.389700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.390029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.414611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:58.417081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:58.450417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 854/1708] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:58 2022] POST /identity/v3/users => generated 303 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:58.454186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 855/1709] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:52:58.456919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:58.473841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd5d4e-59fb-4920-af11-bf0e3a23cfbb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:58.494352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:58.496104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.503596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.504157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.505049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:58.507229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:58.531164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 855/1710] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:58.532297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:58.550626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13017c7a-ab71-4eb2-91de-50d0917a497e service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:58.570180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61de238-fe34-4992-9b94-872246a1e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:58.570891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61de238-fe34-4992-9b94-872246a1e803 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.571705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61de238-fe34-4992-9b94-872246a1e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.571978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61de238-fe34-4992-9b94-872246a1e803 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.572737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61de238-fe34-4992-9b94-872246a1e803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:58.575226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61de238-fe34-4992-9b94-872246a1e803 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:58.602274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 856/1711] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:52:58.603958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:58.627894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848ef6d-f887-4003-bca9-318f48b87422 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:58.644744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:58.645804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.647224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.648570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] PATH_INFO: `/v3/projects/8ca11969faf94ec3956ae61984836b5a/users/7cef9babbfb44d2296143125105c1dc3/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.684057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca11969faf94ec3956ae61984836b5a, user_id=7cef9babbfb44d2296143125105c1dc3, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:58.689788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:58.699417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:58.738245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 856/1712] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:58 2022] PUT /identity/v3/projects/8ca11969faf94ec3956ae61984836b5a/users/7cef9babbfb44d2296143125105c1dc3/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:58.744613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:58.744897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:58.745374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:58.863209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] MFA Rules not processed for user `7cef9babbfb44d2296143125105c1dc3`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:58.864428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:59.153258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 857/1713] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:59.159920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:59.304233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf85f59-b102-407d-882f-403fcb3d651b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:59.316079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917118a-098c-4922-8f9e-f7756d1c05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:59.317046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917118a-098c-4922-8f9e-f7756d1c05ab admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.317472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917118a-098c-4922-8f9e-f7756d1c05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.320082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917118a-098c-4922-8f9e-f7756d1c05ab admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.320995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917118a-098c-4922-8f9e-f7756d1c05ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:59.323174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917118a-098c-4922-8f9e-f7756d1c05ab admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:59.389520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 857/1714] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:52:59 2022] POST /identity/v3/projects => generated 405 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:52:59.389844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:59.403657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41e48a-292f-4e89-8f04-f2e3a17fad6a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:59.522769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:59.523934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.524450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.524871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.525758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:59.528738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:59.542525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9064f9-212c-4ddc-a6f9-e7d3bcdec392 None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.542935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9064f9-212c-4ddc-a6f9-e7d3bcdec392 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.543367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9064f9-212c-4ddc-a6f9-e7d3bcdec392 None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.545183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 858/1715] 10.222.0.86 () {54 vars in 912 bytes} [Wed Jul 20 21:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 21:52:59.554192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.554655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.555253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.573810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 858/1716] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:52:59 2022] POST /identity/v3/users => generated 309 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:52:59.577531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:59.595000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a721a07-6d0a-40c8-92e9-43c3524ff948 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:59.607587 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cfc7c-b114-459b-9624-8f1805c8221b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:59.608427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cfc7c-b114-459b-9624-8f1805c8221b admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.609004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cfc7c-b114-459b-9624-8f1805c8221b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.609154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cfc7c-b114-459b-9624-8f1805c8221b admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.610279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cfc7c-b114-459b-9624-8f1805c8221b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:59.610859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] MFA Rules not processed for user `7f38c9f8b5ea438dbae61a15c1e84e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:59.612178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:59.612297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cfc7c-b114-459b-9624-8f1805c8221b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:59.634149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 859/1717] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:52:59.634746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.636814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.639210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.709450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] MFA Rules not processed for user `fb27ee14314f42cc87e8e6bac2031c70`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:59.711986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:59.738893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 859/1718] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:52:59.742354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:59.769599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f431f-edad-421a-8028-98d47935f267 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:52:59.784784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b36686-298f-4a5a-8e28-a95b981fce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:52:59.784784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36686-298f-4a5a-8e28-a95b981fce77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.785456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36686-298f-4a5a-8e28-a95b981fce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.786545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b36686-298f-4a5a-8e28-a95b981fce77 admin admin] PATH_INFO: `/v3/projects/22be0ad992e04525964abb07cf2fa5d3/users/950bed7930774701a70b1ea6325a8555/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.812337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:52:59.836089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b36686-298f-4a5a-8e28-a95b981fce77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22be0ad992e04525964abb07cf2fa5d3, user_id=950bed7930774701a70b1ea6325a8555, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:52:59.841373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b36686-298f-4a5a-8e28-a95b981fce77 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:52:59.866286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 860/1719] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:52:59 2022] PUT /identity/v3/projects/22be0ad992e04525964abb07cf2fa5d3/users/950bed7930774701a70b1ea6325a8555/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:52:59.868020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:52:59.871038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:52:59.871488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:52:59.919706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 860/1720] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:52:59.921758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:52:59.930092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] MFA Rules not processed for user `60f7f43c067e4b819aeb488e9f09197a`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:52:59.931390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:52:59.933084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b96de8-b039-4e1e-adf0-e11a08a5d6d2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:00.050838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:00.089781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 861/1721] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:52:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:00.094053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:00.100145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.101141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:00.101962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.102316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.102660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.117979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:00.121325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:00.141250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 861/1722] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:52:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:00.163472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:00.237689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6fb11-6aa7-4b72-af44-99b86b6d945f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:00.251203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:00.252576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.253010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.253433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] PATH_INFO: `/v3/projects/9717fbd451374ef08e0779487f95b714` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.256738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9717fbd451374ef08e0779487f95b714)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:00.258898 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:00.332097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fe204-514b-43ce-9561-4d521cc6e940 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:00.341161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:00.342817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.343649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.343762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.344586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:00.348077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:00.363227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 862/1723] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:00.364395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:00.368873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.384109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9717fbd451374ef08e0779487f95b714', 'request_id': 'req-c99993af-0592-402a-9f8c-d5c5ae40c92f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:00.396467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9717fbd451374ef08e0779487f95b714', 'request_id': 'req-c99993af-0592-402a-9f8c-d5c5ae40c92f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:00.415503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9717fbd451374ef08e0779487f95b714', 'request_id': 'req-c99993af-0592-402a-9f8c-d5c5ae40c92f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:00.440392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] The token cache is being invalidate because project 9717fbd451374ef08e0779487f95b714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:00.440778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:00.445623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c99993af-0592-402a-9f8c-d5c5ae40c92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:00.450976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 862/1724] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:00 2022] DELETE /identity/v3/projects/9717fbd451374ef08e0779487f95b714 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:00.454876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f279f-228c-4046-99fb-99fb7a6e791c None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.455332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f279f-228c-4046-99fb-99fb7a6e791c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.455687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f279f-228c-4046-99fb-99fb7a6e791c None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.466870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 863/1725] 10.222.0.86 () {54 vars in 961 bytes} [Wed Jul 20 21:53:00 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:53:00.467853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f279f-228c-4046-99fb-99fb7a6e791c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:00.471930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-433f279f-228c-4046-99fb-99fb7a6e791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.512873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d29a7f-768a-457e-8bd7-d187ea296866 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:00.531756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:00.589068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f279f-228c-4046-99fb-99fb7a6e791c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:00.596086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade59650-67fd-4105-a526-7af658b8302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:00.597748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade59650-67fd-4105-a526-7af658b8302d admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.598282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade59650-67fd-4105-a526-7af658b8302d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.598764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade59650-67fd-4105-a526-7af658b8302d admin admin] PATH_INFO: `/v3/projects/22be0ad992e04525964abb07cf2fa5d3/users/950bed7930774701a70b1ea6325a8555/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.608227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade59650-67fd-4105-a526-7af658b8302d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22be0ad992e04525964abb07cf2fa5d3, user_id=950bed7930774701a70b1ea6325a8555, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:00.613350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade59650-67fd-4105-a526-7af658b8302d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:00.623378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:00.624245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.624533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.624816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.627260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.629722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 864/1726] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:00 2022] PUT /identity/v3/projects/22be0ad992e04525964abb07cf2fa5d3/users/950bed7930774701a70b1ea6325a8555/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:00.632189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893c504-b902-4a62-aac1-309839af3363 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.632744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893c504-b902-4a62-aac1-309839af3363 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.633224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893c504-b902-4a62-aac1-309839af3363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.661277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e893c504-b902-4a62-aac1-309839af3363 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:00.663949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e893c504-b902-4a62-aac1-309839af3363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.683524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:00.713153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:00.714936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d1a0c-2c45-4cfe-b557-36e8d6421c16 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:00.728131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 863/1727] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:00.745751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd68a6-5aaa-4f47-abd2-1367da0f2312 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.746095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd68a6-5aaa-4f47-abd2-1367da0f2312 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.746798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd68a6-5aaa-4f47-abd2-1367da0f2312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.781334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1ffd68a6-5aaa-4f47-abd2-1367da0f2312 None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:00.782590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1ffd68a6-5aaa-4f47-abd2-1367da0f2312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.827001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 865/1728] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:00.827001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.827001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.827001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.871503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] MFA Rules not processed for user `950bed7930774701a70b1ea6325a8555`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:00.872742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:00.957157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 864/1729] 10.222.0.86 () {58 vars in 1058 bytes} [Wed Jul 20 21:53:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:00.960048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:00.960643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:00.962647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:00.989306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 866/1730] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:00.991075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.002142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:01.004328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:01.036224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 865/1731] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:01.037328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.040152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a6c92-7cb3-4636-86c2-839c62954ab9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.043490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:01.058904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.059660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.059946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.060615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.066363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:01.102230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:01.106546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:01.152677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 867/1732] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:01.153665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.211048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:01.213597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95787e10-896f-4aaa-aa08-f52d7d1b572f service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.240073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.242144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.245680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.246106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.247708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:01.249617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:01.251408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a086d78-20e7-448f-bdb1-f8406e408cdf None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.251560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:01.265731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.266420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.266709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.266986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.281057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 868/1733] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:01.291844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.305530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:01.308846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:01.321386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee14961c-4003-47b4-bea4-8f1adfcffd6a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.337346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 866/1734] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:01 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:01.351951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.355541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.356269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.357838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.357915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:01.358104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.358790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:01.364964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:01.500444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 869/1735] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:01 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:01.510279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.548705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ff90e-5187-4d2a-ba30-dfda9e960953 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.691861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:01.714942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e80e6-f6b2-4213-80a3-241e4f746d35 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.721522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.722536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.722976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.723513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.725028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:01.727914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:01.763770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 870/1736] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:01 2022] POST /identity/v3/users => generated 291 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:01.764789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.780422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4401fbd6-45af-4b4b-9e53-7209d47b4bc6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.796921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.797877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.798175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.798459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.799338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:01.801246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:01.822546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 871/1737] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:01.825250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.826986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:01.827562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.828083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.828462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.840913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:01.928667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ea67a-42d4-4bdb-9adf-1f05a5f0b7af admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:01.932949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:01.933587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:01.933889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:01.934158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:01.936304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:02.054650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:02.062474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:02.082585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:02.088170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 867/1738] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 737 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:02.092770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:02.102558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:02.114521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 872/1739] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:01 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:02.157744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab9ec9-5b53-461a-8467-409be1be742c service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:02.180431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdd94b-f3d5-4901-bd4d-e3fabc2ed58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:02.181255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdd94b-f3d5-4901-bd4d-e3fabc2ed58c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:02.183249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdd94b-f3d5-4901-bd4d-e3fabc2ed58c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:02.185385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdd94b-f3d5-4901-bd4d-e3fabc2ed58c admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:02.186981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdd94b-f3d5-4901-bd4d-e3fabc2ed58c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:02.195577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdd94b-f3d5-4901-bd4d-e3fabc2ed58c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:02.237086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 868/1740] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:02.240541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:02.259541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485a705-a1f5-4b5a-81a9-a6264a5e3c6c service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:02.300799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:02.301787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:02.309457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:02.309840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] PATH_INFO: `/v3/projects/d91c8009fd8c4d1b837edc4e506471a1/users/4d3e32de70d4405ab855337199290a57/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:02.378938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91c8009fd8c4d1b837edc4e506471a1, user_id=4d3e32de70d4405ab855337199290a57, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:02.398192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:02.450873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 873/1741] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:02 2022] PUT /identity/v3/projects/d91c8009fd8c4d1b837edc4e506471a1/users/4d3e32de70d4405ab855337199290a57/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:02.452402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:02.452876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:02.453311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:02.501086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:02.527469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:02.529380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:02.580982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] MFA Rules not processed for user `4d3e32de70d4405ab855337199290a57`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:02.589865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:02.826610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:02.833872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 869/1742] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:02.837064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:02.868744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c73c1-2dbd-4b12-bbc9-55c1317f1f45 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:02.883798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:02.884330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:02.884579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:02.884950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:02.893384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:03.123794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:03.125629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4a585-36c0-4d80-885a-a2b189788d5c service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:03.190351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 874/1743] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:03.207505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b092e-1c88-4f90-8810-f0e64fd7b843 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:03.219428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:03.219960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:03.220397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:03.230578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:03.233565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:03.234019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:03.234455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:03.235373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:03.237655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:03.292947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] MFA Rules not processed for user `1a0d4ce0ac4d47c8a184ea0291461379`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:03.308098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:03.378165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 870/1744] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:02 2022] POST /identity/v3/projects => generated 381 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:03.383082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:03.631760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 875/1745] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:03.698748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e9c70-227c-4457-8cfa-02707b9f995c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:03.717267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:03.720358 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:03.721001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:03.721125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:03.721854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:03.741870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:03.844511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 871/1746] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:03 2022] POST /identity/v3/users => generated 297 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:03.848943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:03.880747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19835a68-5fef-4c21-9ace-8212b5f7f412 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:03.902312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:03.905033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:03.906503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:03.906946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:03.910976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:03.918314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:03.953396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 876/1747] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:03.957770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:03.975035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13c443-e629-4359-b45b-4daf1eeb4381 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:03.994078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:03.997016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:03.998285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:04.000296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] PATH_INFO: `/v3/projects/942ff9397e984b0f8a20265774923949/users/590e3616e4d34904aaa804b63176764c/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:04.034820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:04.055414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=942ff9397e984b0f8a20265774923949, user_id=590e3616e4d34904aaa804b63176764c, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:04.063422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:04.106012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 872/1748] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:03 2022] PUT /identity/v3/projects/942ff9397e984b0f8a20265774923949/users/590e3616e4d34904aaa804b63176764c/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:04.110070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:04.143290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6367a2b-ae3b-470f-934c-d66c23db1c16 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:04.332950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ad979-9f0e-44cc-a30b-1d7db7357eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:04.334626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ad979-9f0e-44cc-a30b-1d7db7357eec admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:04.335118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ad979-9f0e-44cc-a30b-1d7db7357eec admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:04.335558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ad979-9f0e-44cc-a30b-1d7db7357eec admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:04.336988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ad979-9f0e-44cc-a30b-1d7db7357eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:04.340904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ad979-9f0e-44cc-a30b-1d7db7357eec admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:04.366608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 877/1749] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:04.371836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:04.392158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d99d8b-567d-48eb-8f6e-9c8d2ab118dc admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:04.429780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:04.434187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:04.434776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:04.436627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] PATH_INFO: `/v3/projects/942ff9397e984b0f8a20265774923949/users/590e3616e4d34904aaa804b63176764c/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:04.463231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] RBAC: Authorizing `identity:create_grant(project_id=942ff9397e984b0f8a20265774923949, user_id=590e3616e4d34904aaa804b63176764c, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:04.491213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:04.527103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 873/1750] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:04 2022] PUT /identity/v3/projects/942ff9397e984b0f8a20265774923949/users/590e3616e4d34904aaa804b63176764c/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:04.537779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:04.538090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:04.538388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:04.644404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] MFA Rules not processed for user `590e3616e4d34904aaa804b63176764c`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:04.648240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:04.732307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:04.931042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 878/1751] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:04.994205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:04.996868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:05.049403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:05.177192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78a43-00c7-4fda-9de8-c8ef4afc455c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:05.196698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:05.198571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:05.201441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd8556-f5a7-4a7b-b5b1-9816dcc50a6b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:05.210414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:05.212855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] PATH_INFO: `/v3/users/d93fa1bf91d54079b3e0a73436d749ce` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:05.213297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8106a5e-1f1b-457b-b467-7c1dba1ad067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:05.213993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106a5e-1f1b-457b-b467-7c1dba1ad067 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:05.214278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106a5e-1f1b-457b-b467-7c1dba1ad067 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:05.214553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8106a5e-1f1b-457b-b467-7c1dba1ad067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:05.215783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d93fa1bf91d54079b3e0a73436d749ce)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:05.219570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:05.236229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8106a5e-1f1b-457b-b467-7c1dba1ad067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:05.237720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8106a5e-1f1b-457b-b467-7c1dba1ad067 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:05.269520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 879/1752] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:53:05.316142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd93fa1bf91d54079b3e0a73436d749ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:05.341822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd93fa1bf91d54079b3e0a73436d749ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:05.354302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd93fa1bf91d54079b3e0a73436d749ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:05.377267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd93fa1bf91d54079b3e0a73436d749ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:05.379863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd93fa1bf91d54079b3e0a73436d749ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:05.395776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 874/1753] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:04 2022] DELETE /identity/v3/users/d93fa1bf91d54079b3e0a73436d749ce => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:05.401536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:05.401939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:05.402084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:05.437319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:05.438877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:05.591000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 880/1754] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:05.638013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:05.640212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:05.874949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025d6e9-6c29-4888-ae05-236fc13def03 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:05.887947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d48b3-f9ab-4dab-9b66-f8d785a9c455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:05.888631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d48b3-f9ab-4dab-9b66-f8d785a9c455 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:05.888917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d48b3-f9ab-4dab-9b66-f8d785a9c455 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:05.889200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d48b3-f9ab-4dab-9b66-f8d785a9c455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:05.909873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:05.923518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d48b3-f9ab-4dab-9b66-f8d785a9c455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:05.931729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d48b3-f9ab-4dab-9b66-f8d785a9c455 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:05.964905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 875/1755] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:06.307159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:06.310620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:06.360609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943d521-46f9-493f-a183-27629aa817e2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:06.382533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:06.384439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:06.384849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:06.385904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] PATH_INFO: `/v3/users/4d3e32de70d4405ab855337199290a57` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:06.388205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d3e32de70d4405ab855337199290a57)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:06.391187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:06.478704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d3e32de70d4405ab855337199290a57'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:06.494193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:06.509636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d3e32de70d4405ab855337199290a57'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:06.519015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d3e32de70d4405ab855337199290a57'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:06.520040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d3e32de70d4405ab855337199290a57'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:06.520869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d3e32de70d4405ab855337199290a57'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:06.540651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 881/1756] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:06 2022] DELETE /identity/v3/users/4d3e32de70d4405ab855337199290a57 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:06.546290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80614b95-1efd-46f9-a922-5cd6a613effc None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:06.546670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80614b95-1efd-46f9-a922-5cd6a613effc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:06.546670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80614b95-1efd-46f9-a922-5cd6a613effc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:06.570131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-80614b95-1efd-46f9-a922-5cd6a613effc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:06.571750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-80614b95-1efd-46f9-a922-5cd6a613effc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:06.668290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 876/1757] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:06.710857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:06.712502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:06.882852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b934ddbb-41f4-4006-b4d4-f7f55e10675d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:06.896635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:06.897483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:06.897870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:06.898246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:06.934787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:06.939252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:06.973712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 882/1758] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:53:07.560930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fefddd-3baa-43f9-b33a-2933fb6db078 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:07.561410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fefddd-3baa-43f9-b33a-2933fb6db078 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:07.561505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fefddd-3baa-43f9-b33a-2933fb6db078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:07.594703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a2fefddd-3baa-43f9-b33a-2933fb6db078 None None] MFA Rules not processed for user `fb27ee14314f42cc87e8e6bac2031c70`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:07.597436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a2fefddd-3baa-43f9-b33a-2933fb6db078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:07.664446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:07.666190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:07.701260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56b1dd-36d2-4c47-9280-1a07eee8585f service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:07.719349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:07.725346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:07.725646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:07.725931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] PATH_INFO: `/v3/projects/948ccdc261db41feb6d0b34bc8c9e1a6` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:07.728642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948ccdc261db41feb6d0b34bc8c9e1a6)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:07.742134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:07.750631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 877/1759] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:07.911463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948ccdc261db41feb6d0b34bc8c9e1a6', 'request_id': 'req-34459816-52c3-4281-901b-af11cba799f1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:07.944402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948ccdc261db41feb6d0b34bc8c9e1a6', 'request_id': 'req-34459816-52c3-4281-901b-af11cba799f1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:07.965013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948ccdc261db41feb6d0b34bc8c9e1a6', 'request_id': 'req-34459816-52c3-4281-901b-af11cba799f1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.004853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] The token cache is being invalidate because project 948ccdc261db41feb6d0b34bc8c9e1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:08.005153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.011889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.020447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 883/1760] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:07 2022] DELETE /identity/v3/projects/948ccdc261db41feb6d0b34bc8c9e1a6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:08.023774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c2c8-9930-4e91-ad40-9a3b1c2fc2af None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:08.024452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c2c8-9930-4e91-ad40-9a3b1c2fc2af None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:08.025219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c2c8-9930-4e91-ad40-9a3b1c2fc2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:08.034891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:08.038288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:08.085743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e466c2c8-9930-4e91-ad40-9a3b1c2fc2af None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:08.087775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e466c2c8-9930-4e91-ad40-9a3b1c2fc2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:08.321368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:08.331578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34459816-52c3-4281-901b-af11cba799f1 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:08.355033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:08.356030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:08.356307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:08.356577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] PATH_INFO: `/v3/projects/d91c8009fd8c4d1b837edc4e506471a1` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:08.364327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91c8009fd8c4d1b837edc4e506471a1)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:08.372705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:08.439807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 878/1761] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:08.442161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af839586-66d3-42a3-a55b-b08dd85feeb5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:08.442456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af839586-66d3-42a3-a55b-b08dd85feeb5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:08.442738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af839586-66d3-42a3-a55b-b08dd85feeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:08.524162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-af839586-66d3-42a3-a55b-b08dd85feeb5 None None] MFA Rules not processed for user `60f7f43c067e4b819aeb488e9f09197a`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:08.531002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-af839586-66d3-42a3-a55b-b08dd85feeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:08.597397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91c8009fd8c4d1b837edc4e506471a1', 'request_id': 'req-6377722a-c00c-4a9e-b8a3-91479dabf361'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.624917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91c8009fd8c4d1b837edc4e506471a1', 'request_id': 'req-6377722a-c00c-4a9e-b8a3-91479dabf361'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.637233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:08.651990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91c8009fd8c4d1b837edc4e506471a1', 'request_id': 'req-6377722a-c00c-4a9e-b8a3-91479dabf361'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.673652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 879/1762] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:08.677882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:08.679489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:08.681629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:08.685845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] The token cache is being invalidate because project d91c8009fd8c4d1b837edc4e506471a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:08.686350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.692239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:08.696237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 884/1763] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:08 2022] DELETE /identity/v3/projects/d91c8009fd8c4d1b837edc4e506471a1 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:08.699995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:08.707119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:08.745223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:08.746360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:08.929162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377722a-c00c-4a9e-b8a3-91479dabf361 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:08.946388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:08.948770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:08.952501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:08.952501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] PATH_INFO: `/v3/users/590e3616e4d34904aaa804b63176764c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:08.957422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590e3616e4d34904aaa804b63176764c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:08.966807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:08.980098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 880/1764] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:08.991552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:09.007157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a89988-7f53-4890-98b4-4498a9fdf7ad None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:09.023589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:09.024807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.025276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.025724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.026656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:09.027177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:09.029192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:09.066323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 881/1765] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:53:09.070485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:09.106059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd014a3-712c-4dee-a60e-3e22cb30236f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:09.115823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:09.116060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590e3616e4d34904aaa804b63176764c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:09.116514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.116800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.117083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.118253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:09.120392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:09.168698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590e3616e4d34904aaa804b63176764c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:09.181110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 882/1766] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:09 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:53:09.190851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:09.236327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590e3616e4d34904aaa804b63176764c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:09.237885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590e3616e4d34904aaa804b63176764c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:09.238780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c0bab965-f8b9-4206-881b-e5d3620c1f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590e3616e4d34904aaa804b63176764c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:09.283743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 885/1767] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:08 2022] DELETE /identity/v3/users/590e3616e4d34904aaa804b63176764c => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:09.286338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.287440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.288500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.308079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:09.395884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843519-41be-41c9-8a76-37726408d069 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:09.436794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:09.442164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:09.606703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:09.607994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.608435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.608940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.610299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:09.613307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:09.625891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:09.668093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 883/1768] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:09 2022] POST /identity/v3/users => generated 287 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:09.672427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:09.675902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:09.701172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74afc0-1453-4863-a2f1-fecc419a0845 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:09.709234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:09.711512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.714237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.714781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.717639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 886/1769] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:53:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:09.724449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:09.727206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:09.746156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 884/1770] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:09.746901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:09.774351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37e46d-1ddf-4996-8772-9da0604c32a1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:09.783992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be8a80-a6b6-496c-962c-bc0d4d6fd1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:09.784801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be8a80-a6b6-496c-962c-bc0d4d6fd1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.785159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be8a80-a6b6-496c-962c-bc0d4d6fd1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.785507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be8a80-a6b6-496c-962c-bc0d4d6fd1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.786329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be8a80-a6b6-496c-962c-bc0d4d6fd1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:09.789111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be8a80-a6b6-496c-962c-bc0d4d6fd1e5 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:09.828489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 887/1771] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:09.829974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:09.844649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73babf02-d476-4480-8c16-92681f5675b1 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:09.851117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:09.853427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.854427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.855073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] PATH_INFO: `/v3/projects/d6a16b01609947638fea7376866fe19c/users/364d8579a2484c75ba039fc319f86ac7/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:09.878872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a16b01609947638fea7376866fe19c, user_id=364d8579a2484c75ba039fc319f86ac7, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:09.885049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:09.948662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 885/1772] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:09 2022] PUT /identity/v3/projects/d6a16b01609947638fea7376866fe19c/users/364d8579a2484c75ba039fc319f86ac7/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:09.960666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:09.961046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:09.961329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:10.014318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:10.016334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:10.052018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] MFA Rules not processed for user `364d8579a2484c75ba039fc319f86ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:10.057751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:10.211242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 888/1773] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:10.218315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:10.301383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede977c2-6e24-4c71-abe9-4a3372dbcbd5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:10.311558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:10.312298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:10.312631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:10.313274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:10.317309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:10.325651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b08c4-9243-4162-90a9-10da976f1f14 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:10.334070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:10.334962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:10.335671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:10.335958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:10.336675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:10.339305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:10.388480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 889/1774] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:10 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:10.396267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:10.460124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:10.465266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:10.488217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 886/1775] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:53:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:53:10.648604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a44451-3538-4c9c-bda7-fa7a2161ed27 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:10.684407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:10.685123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:10.685453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:10.691284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:10.700629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:10.719695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:10.840222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 890/1776] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:10 2022] POST /identity/v3/users => generated 293 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:10.846178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:10.870137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de6e1-5f23-4c58-8831-3be01770b2db service swift] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:10.878741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:10.881355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:10.883700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:10.884764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:10.887290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:10.889982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:10.925405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 887/1777] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:10.926368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:10.940389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cbebd-be11-4dca-b860-4182d910d212 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:10.957604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:10.958790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:10.959613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:10.960117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] PATH_INFO: `/v3/projects/8b88faf444ac472fb71fce8a890fa346/users/efbd775a098240a1b2773c5ed700caad/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:11.005733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b88faf444ac472fb71fce8a890fa346, user_id=efbd775a098240a1b2773c5ed700caad, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:11.012176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:11.058091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 891/1778] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:10 2022] PUT /identity/v3/projects/8b88faf444ac472fb71fce8a890fa346/users/efbd775a098240a1b2773c5ed700caad/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:11.064115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:11.119860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f1a7d-106b-493d-8f60-33825e99bb73 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:11.244103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:11.405682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30914bd-806c-4fe8-a86f-a8ed8a44cb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:11.406576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30914bd-806c-4fe8-a86f-a8ed8a44cb22 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:11.406867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30914bd-806c-4fe8-a86f-a8ed8a44cb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:11.416499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30914bd-806c-4fe8-a86f-a8ed8a44cb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:11.416499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30914bd-806c-4fe8-a86f-a8ed8a44cb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:11.418152 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30914bd-806c-4fe8-a86f-a8ed8a44cb22 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:11.454099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:11.454618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 888/1779] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:11.473919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e646ec-5d3b-4033-b46b-ef069725d6cc admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:11.500513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee9f34-4b30-4edb-beb4-782d823c8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:11.501533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee9f34-4b30-4edb-beb4-782d823c8c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:11.509129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee9f34-4b30-4edb-beb4-782d823c8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:11.509729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee9f34-4b30-4edb-beb4-782d823c8c27 admin admin] PATH_INFO: `/v3/projects/8b88faf444ac472fb71fce8a890fa346/users/efbd775a098240a1b2773c5ed700caad/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:11.523306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee9f34-4b30-4edb-beb4-782d823c8c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b88faf444ac472fb71fce8a890fa346, user_id=efbd775a098240a1b2773c5ed700caad, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:11.544621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee9f34-4b30-4edb-beb4-782d823c8c27 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:11.600551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 892/1780] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:11 2022] PUT /identity/v3/projects/8b88faf444ac472fb71fce8a890fa346/users/efbd775a098240a1b2773c5ed700caad/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:11.615500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:11.615826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:11.616345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:11.686635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] MFA Rules not processed for user `efbd775a098240a1b2773c5ed700caad`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:11.688441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:11.819918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:11.820525 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:11.820611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:11.902759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] MFA Rules not processed for user `7f38c9f8b5ea438dbae61a15c1e84e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:11.907586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:11.951448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 889/1781] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:11.958996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:11.960795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:12.121720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 893/1782] 10.222.0.74 () {58 vars in 1015 bytes} [Wed Jul 20 21:53:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:12.191212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:12.192972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:12.207447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23f174-a217-4d2a-8876-2a240efb5557 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:12.211981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3560b4e-6eeb-4c22-8a1a-7b37170f0fe2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:12.216777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:12.217482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:12.217764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:12.218041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:12.225700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:12.227555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:12.227914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:12.230117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:12.259901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:12.261783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:12.269312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:12.270860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:12.295166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 890/1783] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:53:12.320782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 894/1784] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:12.608883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:12.610726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:12.657633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:12.660180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:12.687544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f9ba5-7b40-41ae-b853-f3220b4ea19d service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:12.699558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8fab03-66fd-45c8-be92-cc20cf0bbfce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:12.701590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab03-66fd-45c8-be92-cc20cf0bbfce service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:12.702580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab03-66fd-45c8-be92-cc20cf0bbfce service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:12.703621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8fab03-66fd-45c8-be92-cc20cf0bbfce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:12.796356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:12.798346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:12.862527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763a017-e6d9-4f40-af5f-c9a0f50cf6ed service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:12.880223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fab03-66fd-45c8-be92-cc20cf0bbfce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:12.880930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:12.881937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:12.882464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:12.883631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] PATH_INFO: `/v3/users/7cef9babbfb44d2296143125105c1dc3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:12.885457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cef9babbfb44d2296143125105c1dc3)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:12.887653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:12.899705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8fab03-66fd-45c8-be92-cc20cf0bbfce service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:12.945630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 895/1785] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:13.017081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cef9babbfb44d2296143125105c1dc3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:13.095002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cef9babbfb44d2296143125105c1dc3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:13.124166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cef9babbfb44d2296143125105c1dc3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:13.142395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cef9babbfb44d2296143125105c1dc3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:13.143107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cef9babbfb44d2296143125105c1dc3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:13.158996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 891/1786] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:12 2022] DELETE /identity/v3/users/7cef9babbfb44d2296143125105c1dc3 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:13.160305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:13.160654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:13.160939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:13.216598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:13.217776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:13.473427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 896/1787] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:13.551544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:13.555099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:13.726345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87ecc-4c97-4522-8b55-914cfe9b6708 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:13.736190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f982b-c039-4d42-ab70-b5cca20bb385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:13.737415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f982b-c039-4d42-ab70-b5cca20bb385 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:13.737968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f982b-c039-4d42-ab70-b5cca20bb385 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:13.738425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f982b-c039-4d42-ab70-b5cca20bb385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:13.764949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f982b-c039-4d42-ab70-b5cca20bb385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:13.767511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f982b-c039-4d42-ab70-b5cca20bb385 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:13.787176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 892/1788] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:14.370255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:14.371876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:14.407326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd7415-2421-432f-ae97-2f1e1f803d55 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:14.422316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:14.423762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:14.424292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:14.424772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] PATH_INFO: `/v3/projects/942ff9397e984b0f8a20265774923949` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:14.437024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] RBAC: Authorizing `identity:delete_project(project_id=942ff9397e984b0f8a20265774923949)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:14.440684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:14.539213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942ff9397e984b0f8a20265774923949', 'request_id': 'req-7f72b815-fbdc-42d9-908a-06b16b4ed34f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:14.549456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942ff9397e984b0f8a20265774923949', 'request_id': 'req-7f72b815-fbdc-42d9-908a-06b16b4ed34f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:14.559813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942ff9397e984b0f8a20265774923949', 'request_id': 'req-7f72b815-fbdc-42d9-908a-06b16b4ed34f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:14.585475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] The token cache is being invalidate because project 942ff9397e984b0f8a20265774923949 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:14.585961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:14.586881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7f72b815-fbdc-42d9-908a-06b16b4ed34f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:14.592390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 897/1789] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:14 2022] DELETE /identity/v3/projects/942ff9397e984b0f8a20265774923949 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:14.594426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:14.594948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:14.595567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:14.661308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:14.664099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:14.872172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 893/1790] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:14.881883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:14.883637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:14.883637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:14.952132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:14.963717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:15.000361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 898/1791] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:15.005085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:15.031962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede75bd-ec6b-418f-88ec-c68ef1f630f4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:15.053601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:15.054829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:15.055339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:15.055728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:15.056619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:15.058283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:15.060735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:15.096081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:15.123952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 894/1792] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:15.124242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:15.140997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c1bc5-57ae-4e77-9ebf-85add7e0c4c4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:15.176207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:15.178933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:15.180410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:15.185130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:15.188082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:15.192394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:15.319656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 899/1793] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:15 2022] POST /identity/v3/projects => generated 385 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:15.322681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:15.356602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aa0b6-26fc-4682-80e9-c2debb71f4a5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:15.637430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:15.638513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:15.638513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:15.638922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:15.639739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:15.641711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:15.731274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:15.745771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 895/1794] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:15 2022] POST /identity/v3/users => generated 293 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:15.750356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5834-6495-47cf-ac8f-d47e40e22a55 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:15.780867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:15.781889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:15.785059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:15.785624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:15.788767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:15.801472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:15.852732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 900/1795] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:15.858029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:15.875867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099daed-087f-4fe9-900a-665daf887ff6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:15.897038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e82db-7e0e-42c6-b478-555af9e07d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:15.898083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e82db-7e0e-42c6-b478-555af9e07d67 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:15.898601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e82db-7e0e-42c6-b478-555af9e07d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:15.899092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e82db-7e0e-42c6-b478-555af9e07d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:15.900069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e82db-7e0e-42c6-b478-555af9e07d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:15.902901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71e82db-7e0e-42c6-b478-555af9e07d67 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:15.940660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:15.951735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 896/1796] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:15.963919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67f9b8-b21e-41bf-b197-698bbcce5992 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:15.980032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:15.981317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:15.981873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:15.982346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] PATH_INFO: `/v3/projects/49dd21070087426dac889da9cb83ec51/users/a9e52a3cbca840fbb8b32207dac86de6/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:16.003706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:16.016945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49dd21070087426dac889da9cb83ec51, user_id=a9e52a3cbca840fbb8b32207dac86de6, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:16.023370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:16.054679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 901/1797] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:15 2022] PUT /identity/v3/projects/49dd21070087426dac889da9cb83ec51/users/a9e52a3cbca840fbb8b32207dac86de6/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:16.066227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e06bc-94ea-44bd-a15b-5cd335a2dfec None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:16.066605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e06bc-94ea-44bd-a15b-5cd335a2dfec None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:16.067024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e06bc-94ea-44bd-a15b-5cd335a2dfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:16.156098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1b5e06bc-94ea-44bd-a15b-5cd335a2dfec None None] MFA Rules not processed for user `a9e52a3cbca840fbb8b32207dac86de6`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:16.157444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1b5e06bc-94ea-44bd-a15b-5cd335a2dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:16.403566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 897/1798] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:16.416375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:16.418299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:16.525142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531bc16-51e6-4d23-82c9-50d2ce542607 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:16.526357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531bc16-51e6-4d23-82c9-50d2ce542607 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:16.526750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531bc16-51e6-4d23-82c9-50d2ce542607 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:16.533851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 898/1799] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:53:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:53:16.601094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b4b87-b4ed-4d8d-96cd-7467e58eb3a7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:16.608090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:16.608927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:16.609412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:16.610168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:16.669294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303b7ff-38cf-40f8-8a2e-987c7e8565ed None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:16.669774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303b7ff-38cf-40f8-8a2e-987c7e8565ed None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:16.670268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303b7ff-38cf-40f8-8a2e-987c7e8565ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:16.672366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:16.673919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:16.688640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 902/1800] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:16.716652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:16.735105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3303b7ff-38cf-40f8-8a2e-987c7e8565ed None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:16.739191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3303b7ff-38cf-40f8-8a2e-987c7e8565ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:16.851433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:16.892138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 899/1801] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:53:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:17.013806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f7af6-c8e9-42c3-a60b-7a6b84447166 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:17.053849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:17.055029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:17.057887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:17.059515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] PATH_INFO: `/v3/users/a9e52a3cbca840fbb8b32207dac86de6` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:17.068585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e52a3cbca840fbb8b32207dac86de6)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:17.073369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:17.313529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e52a3cbca840fbb8b32207dac86de6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.375719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e52a3cbca840fbb8b32207dac86de6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.400381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e52a3cbca840fbb8b32207dac86de6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.402730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e52a3cbca840fbb8b32207dac86de6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.404167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e52a3cbca840fbb8b32207dac86de6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.445870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 903/1802] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:16 2022] DELETE /identity/v3/users/a9e52a3cbca840fbb8b32207dac86de6 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:17.448386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:17.449182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:17.449954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:17.487894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:17.489610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:17.512751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:17.515768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:17.539045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:17.766194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9641081-0e7e-4988-920e-d86cef94ef8c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:17.791589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:17.792500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:17.792789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:17.793072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] PATH_INFO: `/v3/projects/8ca11969faf94ec3956ae61984836b5a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:17.805174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca11969faf94ec3956ae61984836b5a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:17.806116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 900/1803] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:17.808196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:17.918228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca11969faf94ec3956ae61984836b5a', 'request_id': 'req-a454e83a-5ae7-4e07-8c4f-d923611c5962'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.936449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca11969faf94ec3956ae61984836b5a', 'request_id': 'req-a454e83a-5ae7-4e07-8c4f-d923611c5962'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.960560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca11969faf94ec3956ae61984836b5a', 'request_id': 'req-a454e83a-5ae7-4e07-8c4f-d923611c5962'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.986538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] The token cache is being invalidate because project 8ca11969faf94ec3956ae61984836b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:17.986843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:17.991875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:18.004065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 904/1804] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:17 2022] DELETE /identity/v3/projects/8ca11969faf94ec3956ae61984836b5a => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:18.009605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:18.045565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a865793-4104-4e97-a431-8ebb69d85637 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:18.067703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:18.070021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:18.318039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e83a-5ae7-4e07-8c4f-d923611c5962 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:18.330561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:18.331311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:18.334989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:18.335305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:18.350772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:18.351485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:18.357231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:18.360091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:18.360623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] PATH_INFO: `/v3/users/950bed7930774701a70b1ea6325a8555` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:18.366645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=950bed7930774701a70b1ea6325a8555)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:18.387923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:18.411316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:18.422538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:18.475446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 905/1805] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:18.559361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '950bed7930774701a70b1ea6325a8555'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:18.593752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '950bed7930774701a70b1ea6325a8555'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:18.620040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '950bed7930774701a70b1ea6325a8555'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:18.648167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '950bed7930774701a70b1ea6325a8555'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:18.648833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '950bed7930774701a70b1ea6325a8555'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:18.658827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 901/1806] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:18 2022] DELETE /identity/v3/users/950bed7930774701a70b1ea6325a8555 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:18.992191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:18.995401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:19.102784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:19.214143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec677-9de4-4930-8a00-48acc0c260a8 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:19.441391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:19.442090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:19.442376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:19.443281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:19.452255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:19.478964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:19.480646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04959e-f084-4d2b-a5ef-913834190301 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:19.501280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 906/1807] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:19.734507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:19.736305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:19.777206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:19.858197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed07fa5-776a-4737-b171-2ec64d8c8dd2 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:19.868634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:19.869402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:19.871123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:19.871123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] PATH_INFO: `/v3/projects/22be0ad992e04525964abb07cf2fa5d3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:19.872644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=22be0ad992e04525964abb07cf2fa5d3)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:19.874683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:19.970912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22be0ad992e04525964abb07cf2fa5d3', 'request_id': 'req-1c719470-dd1e-4dd2-a949-f4d42d3db2af'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:19.985079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22be0ad992e04525964abb07cf2fa5d3', 'request_id': 'req-1c719470-dd1e-4dd2-a949-f4d42d3db2af'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:19.989300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22be0ad992e04525964abb07cf2fa5d3', 'request_id': 'req-1c719470-dd1e-4dd2-a949-f4d42d3db2af'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.009069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] The token cache is being invalidate because project 22be0ad992e04525964abb07cf2fa5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:20.010607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.011834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1c719470-dd1e-4dd2-a949-f4d42d3db2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.013791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 902/1808] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:19 2022] DELETE /identity/v3/projects/22be0ad992e04525964abb07cf2fa5d3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:20.018640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.019255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.019533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.042080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:20.043641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.121833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 907/1809] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:20.132738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.133150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.133518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.164167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:20.165012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.165903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.167778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.175651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 903/1810] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:20.180309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.192317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa478b46-221b-4062-841f-7f4883a05337 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.192747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4436d50-0ed9-4565-b975-e10206cc3b7a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.204455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.205893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.206363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.206800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] PATH_INFO: `/v3/projects/49dd21070087426dac889da9cb83ec51` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.206924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.208397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.208733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] RBAC: Authorizing `identity:delete_project(project_id=49dd21070087426dac889da9cb83ec51)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.208857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.209223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.210368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:20.210787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.213201 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.213363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.243881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 904/1811] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:20.251912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.288291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49dd21070087426dac889da9cb83ec51', 'request_id': 'req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.288737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa9971-b88d-4b06-a75e-3f4bdd0c5bd5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.297306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49dd21070087426dac889da9cb83ec51', 'request_id': 'req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.314950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49dd21070087426dac889da9cb83ec51', 'request_id': 'req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.316892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:20.331944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] The token cache is being invalidate because project 49dd21070087426dac889da9cb83ec51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:20.332384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.333720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-df0a61b5-e2d1-44a8-bfeb-392c766dc49d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:20.336234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 908/1812] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:20 2022] DELETE /identity/v3/projects/49dd21070087426dac889da9cb83ec51 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:20.340954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.341342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.341754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.365161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:20.366318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.405481 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04cf99-f438-45f7-b5f7-822a2031ed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.406171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04cf99-f438-45f7-b5f7-822a2031ed7e admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.406454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04cf99-f438-45f7-b5f7-822a2031ed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.406747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04cf99-f438-45f7-b5f7-822a2031ed7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.407496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04cf99-f438-45f7-b5f7-822a2031ed7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.409496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04cf99-f438-45f7-b5f7-822a2031ed7e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.439389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 909/1813] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:20.445975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 905/1814] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:20.448417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.449002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.449312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.449585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.450206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.464647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81ffb8-ff3c-4376-890e-99467137993a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.468219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:20.469469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.480022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:20.534795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.535608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.535962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.536310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.537829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.539819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.546885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 906/1815] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:20.551341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.561451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d3f1e-b0ed-42b7-bf88-4fa4047cd1a8 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.562196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 910/1816] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:20.566354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.566549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.567252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.567540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.568053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.568971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:20.569287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.570907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.576867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75c390-2579-4796-b336-a7a328a23297 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.583811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.587691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 907/1817] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:53:20.587835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.587835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.587835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.587835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.587835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.588698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.598113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 911/1818] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:20.599816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3be22f-e475-4726-a0e3-3e368160ae50 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.602705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.604997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.605705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.605995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.606277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.606992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.608961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.615361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab91cd6-9102-496f-958b-af75c77a8c6b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.620768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.621534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.621883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.622227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.623030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.624694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.641914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 912/1819] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:20.643254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.652402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 908/1820] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:20.656753 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17dbf-4ef8-4bc0-a103-bd66978a77f4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.657046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.737298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a71657-4492-4eb3-8e0a-7612f17dc3d7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.743795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.744596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.745134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.745291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.746081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.748161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.748400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a495766-c62b-44cc-9bed-0df18d198d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.749354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a495766-c62b-44cc-9bed-0df18d198d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.749705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a495766-c62b-44cc-9bed-0df18d198d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.750052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a495766-c62b-44cc-9bed-0df18d198d01 admin admin] PATH_INFO: `/v3/projects/d742482a89c84e4591310532a7e1d1db/users/45c459f4710f43f99294d91084e9b995/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.752743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:20.773425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a495766-c62b-44cc-9bed-0df18d198d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d742482a89c84e4591310532a7e1d1db, user_id=45c459f4710f43f99294d91084e9b995, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.775676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 909/1821] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:20.776208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a495766-c62b-44cc-9bed-0df18d198d01 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.779707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.791858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff32c26-8ac3-4abe-b71f-d877c9f75a96 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.805564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 913/1822] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:20 2022] PUT /identity/v3/projects/d742482a89c84e4591310532a7e1d1db/users/45c459f4710f43f99294d91084e9b995/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:20.807280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.807638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.807984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.852873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] MFA Rules not processed for user `45c459f4710f43f99294d91084e9b995`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:20.854581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:20.900536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.901227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.901511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.901785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.902536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.904373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.916674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 910/1823] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:20.921189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.931461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:20.936380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ecd888-5d8a-404c-936b-cc971b667d20 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.944437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.945303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.945624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.945907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.946654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.949174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.952393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 914/1824] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:20.960069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.970171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 911/1825] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:20.978573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33150b2c-6514-4cf9-a58a-b7209c4bbf19 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:20.980243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:20.989877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:20.990964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:20.991392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:20.991845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:20.992584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:20.995834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:20.997346 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1281-7704-4fa3-8610-175e87cc2ee6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:21.005854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b75f7-d374-4353-a4f3-811ba190071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:21.007781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b75f7-d374-4353-a4f3-811ba190071a admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:21.008378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b75f7-d374-4353-a4f3-811ba190071a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:21.008914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b75f7-d374-4353-a4f3-811ba190071a admin admin] PATH_INFO: `/v3/projects/762fed7c2947439897af16bd2d6b84db/users/8115ef190ebb45d89b28ef235aff2e59/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:21.038058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 915/1826] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:20 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:53:21.039337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:21.040898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b75f7-d374-4353-a4f3-811ba190071a admin admin] RBAC: Authorizing `identity:create_grant(project_id=762fed7c2947439897af16bd2d6b84db, user_id=8115ef190ebb45d89b28ef235aff2e59, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:21.044237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b75f7-d374-4353-a4f3-811ba190071a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:21.088598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 912/1827] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:20 2022] PUT /identity/v3/projects/762fed7c2947439897af16bd2d6b84db/users/8115ef190ebb45d89b28ef235aff2e59/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:21.104355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:21.104787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:21.105266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:21.155291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] MFA Rules not processed for user `8115ef190ebb45d89b28ef235aff2e59`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:21.157375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:21.185465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd554f46-a76b-472c-8b3b-1f805e80ddae admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:21.301664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:21.315431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:21.316735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:21.318281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:21.319196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:21.320497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:21.330966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 913/1828] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:21.332089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:21.334028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:21.351176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:21.409893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 916/1829] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:21 2022] POST /identity/v3/users => generated 297 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:21.411001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:21.448665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f58fd9c-3343-4eb2-ac77-a23cb1697db9 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:21.457158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:21.458574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:21.460238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:21.462629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:21.463915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:21.471036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:21.485751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea086193-aaf1-4b19-b7e5-fc6c03fd487f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:21.494535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:21.495777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:21.498339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:21.498685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:21.500292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 917/1830] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:21.509468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:21.536137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:21.538575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:21.543869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a81277-3a64-4749-a132-aa3970e4a40c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:21.553772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:21.556696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:21.557251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:21.557793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] PATH_INFO: `/v3/projects/30ca4be728194b9ca49781180e6c3b44/users/ca93a16c0305455ab9a7d9874b030b92/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:21.571676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 914/1831] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:21.594603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ca4be728194b9ca49781180e6c3b44, user_id=ca93a16c0305455ab9a7d9874b030b92, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:21.603449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:21.688085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 918/1832] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:21 2022] PUT /identity/v3/projects/30ca4be728194b9ca49781180e6c3b44/users/ca93a16c0305455ab9a7d9874b030b92/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:21.692950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:21.750919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e3048-cda0-4d1d-b38c-26f0be95df1b service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:22.021837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03777f56-1cb8-4dbb-a52f-d0ac47f5672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:22.022575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03777f56-1cb8-4dbb-a52f-d0ac47f5672c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:22.022891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03777f56-1cb8-4dbb-a52f-d0ac47f5672c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:22.023141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03777f56-1cb8-4dbb-a52f-d0ac47f5672c admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:22.023918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03777f56-1cb8-4dbb-a52f-d0ac47f5672c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:22.025871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03777f56-1cb8-4dbb-a52f-d0ac47f5672c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:22.031865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:22.050586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:22.056350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 915/1833] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:22.070103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200074-4fc9-4f5d-a589-063ba04ab9d0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:22.102113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:22.103402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:22.103696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:22.103991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] PATH_INFO: `/v3/projects/30ca4be728194b9ca49781180e6c3b44/users/ca93a16c0305455ab9a7d9874b030b92/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:22.113027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30ca4be728194b9ca49781180e6c3b44, user_id=ca93a16c0305455ab9a7d9874b030b92, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:22.119480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:22.153351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 919/1834] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:22 2022] PUT /identity/v3/projects/30ca4be728194b9ca49781180e6c3b44/users/ca93a16c0305455ab9a7d9874b030b92/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:22.164195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:22.164488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:22.164772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:22.220646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] MFA Rules not processed for user `ca93a16c0305455ab9a7d9874b030b92`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:22.222675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:22.393786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 916/1835] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:22.447713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:22.672405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcc2c9-64d8-43b9-b107-c98e760f0be0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:22.683288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:22.684669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:22.685061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:22.685440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:22.720318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:22.722190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:22.747777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 920/1836] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:22.892645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:22.914832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358063f-8ac1-4792-9f0f-0e55382d90d7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:22.925681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ccc40-6f50-423a-b868-e3516cfa79b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:22.932380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ccc40-6f50-423a-b868-e3516cfa79b5 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:22.935542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ccc40-6f50-423a-b868-e3516cfa79b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:22.941194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ccc40-6f50-423a-b868-e3516cfa79b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:23.114454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ccc40-6f50-423a-b868-e3516cfa79b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:23.117040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ccc40-6f50-423a-b868-e3516cfa79b5 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:23.143986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 917/1837] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:24.077402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:24.378909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d042a-3dd5-49fa-995f-0f22bae8b5ad service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:24.437288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:24.438773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:24.439277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:24.439758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] PATH_INFO: `/v3/users/45c459f4710f43f99294d91084e9b995` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:24.450523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c459f4710f43f99294d91084e9b995)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:24.454481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:24.647504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:24.651913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c459f4710f43f99294d91084e9b995'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:24.685633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c459f4710f43f99294d91084e9b995'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:24.709074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c459f4710f43f99294d91084e9b995'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:24.710816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c459f4710f43f99294d91084e9b995'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:24.712034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c459f4710f43f99294d91084e9b995'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:24.762792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 921/1838] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:24 2022] DELETE /identity/v3/users/45c459f4710f43f99294d91084e9b995 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:24.768807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:24.769305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:24.769379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:24.812424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:24.813786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:24.920608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 918/1839] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:24.933415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:24.934972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:25.007435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbad85-3e9f-44b7-baca-f75bce67a6ad admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:25.011931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:25.012766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:25.013108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:25.013449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:25.024156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:25.025614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:25.037285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 922/1840] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:25.340568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:25.342233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:25.371837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d1f0b-1615-4fc0-a143-b8605ad23a68 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:25.378234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:25.379192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:25.379512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:25.379793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] PATH_INFO: `/v3/projects/d742482a89c84e4591310532a7e1d1db` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:25.381547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d742482a89c84e4591310532a7e1d1db)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:25.383599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:25.406342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:25.442043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd742482a89c84e4591310532a7e1d1db', 'request_id': 'req-fa8beb62-b22e-4ff8-a564-a43685c1626a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.449838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd742482a89c84e4591310532a7e1d1db', 'request_id': 'req-fa8beb62-b22e-4ff8-a564-a43685c1626a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.456298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd742482a89c84e4591310532a7e1d1db', 'request_id': 'req-fa8beb62-b22e-4ff8-a564-a43685c1626a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.471788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] The token cache is being invalidate because project d742482a89c84e4591310532a7e1d1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:25.472853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.473546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.475236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 919/1841] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:25 2022] DELETE /identity/v3/projects/d742482a89c84e4591310532a7e1d1db => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:25.479164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:25.489577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c256f5-794e-43ad-a3ac-aa320f927b0f service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:25.551595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:25.553086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:25.553367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:25.553644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] PATH_INFO: `/v3/users/ca93a16c0305455ab9a7d9874b030b92` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:25.554937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca93a16c0305455ab9a7d9874b030b92)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:25.556943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:25.619474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca93a16c0305455ab9a7d9874b030b92'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.633218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca93a16c0305455ab9a7d9874b030b92'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.634679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:25.644381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca93a16c0305455ab9a7d9874b030b92'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.645387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca93a16c0305455ab9a7d9874b030b92'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.646080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8b0c64f6-7b26-442a-9aec-48a1d3d3e872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca93a16c0305455ab9a7d9874b030b92'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:25.663336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 923/1842] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:25 2022] DELETE /identity/v3/users/ca93a16c0305455ab9a7d9874b030b92 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:25.999380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.001081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:26.026316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8beb62-b22e-4ff8-a564-a43685c1626a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.105697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.106597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.106881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.107223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] PATH_INFO: `/v3/projects/30ca4be728194b9ca49781180e6c3b44` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.108540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30ca4be728194b9ca49781180e6c3b44)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.110632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.169628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30ca4be728194b9ca49781180e6c3b44', 'request_id': 'req-44e1922d-a0ab-49f0-b450-83553bb0aee7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:26.179074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30ca4be728194b9ca49781180e6c3b44', 'request_id': 'req-44e1922d-a0ab-49f0-b450-83553bb0aee7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:26.184339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30ca4be728194b9ca49781180e6c3b44', 'request_id': 'req-44e1922d-a0ab-49f0-b450-83553bb0aee7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:26.197525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] The token cache is being invalidate because project 30ca4be728194b9ca49781180e6c3b44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:26.197809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:26.198475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-44e1922d-a0ab-49f0-b450-83553bb0aee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:26.200109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 920/1843] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:25 2022] DELETE /identity/v3/projects/30ca4be728194b9ca49781180e6c3b44 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:26.206391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.206677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.206953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.228244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:26.229461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:26.294980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 924/1844] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:26.302963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.303283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.307406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.322181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:26.323424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:26.331587 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 921/1845] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:26.336612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.346783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ff84-d49e-4045-8273-e2d6a010a770 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.352129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.352732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.353010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.353304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.354032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:26.354334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.356880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.368824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 925/1846] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:26.373543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.383192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3436e8-dcb6-4fee-917f-6ea837292440 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.387859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.388521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.388803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.389076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.389777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.391746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.423445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 922/1847] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:26 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:26.425633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.437502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24255c-1026-41ad-aa3e-6c9bc056e294 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.498706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.499628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.499726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.499943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.503970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.503970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.523941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 926/1848] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:26 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:26.528364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.540388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99ed4c-55cb-42e8-b18c-4fd69d876be5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.545199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.545914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.546362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.546646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.547544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.549173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.550691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:26.562943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 923/1849] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:26.566538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.577206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d639e-3aec-4d95-8699-63080e54516f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.581963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df0693-08ab-47b8-a726-5ed567542aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.582625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df0693-08ab-47b8-a726-5ed567542aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.582907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df0693-08ab-47b8-a726-5ed567542aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.584146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df0693-08ab-47b8-a726-5ed567542aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.584993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df0693-08ab-47b8-a726-5ed567542aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.586617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df0693-08ab-47b8-a726-5ed567542aa4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.596938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 927/1850] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:26.602680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.614380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f77850-a294-4156-8863-064630baf3a4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.622826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.623772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.624057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.624336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] PATH_INFO: `/v3/projects/2b546ac5537046a3a9b17bb33915f440/users/6b3a0451e20f42a38ba6dc524a097363/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.643295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b546ac5537046a3a9b17bb33915f440, user_id=6b3a0451e20f42a38ba6dc524a097363, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.646726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.664819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 924/1851] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:26 2022] PUT /identity/v3/projects/2b546ac5537046a3a9b17bb33915f440/users/6b3a0451e20f42a38ba6dc524a097363/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:26.670249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cfa74-d5d9-4ac5-ba4d-bf8bc4d6d6b2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.670544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cfa74-d5d9-4ac5-ba4d-bf8bc4d6d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.670820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cfa74-d5d9-4ac5-ba4d-bf8bc4d6d6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.689964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7f6cfa74-d5d9-4ac5-ba4d-bf8bc4d6d6b2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:26.691235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7f6cfa74-d5d9-4ac5-ba4d-bf8bc4d6d6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:26.744484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:26.763824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 928/1852] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:26.774329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:26.776363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:26.848950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148572c3-23d8-4e42-a257-1237b5399f76 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:26.853851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:26.854535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:26.854821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:26.855119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:26.866807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:26.868588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:26.882022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 925/1853] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:27.596828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:27.597371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:27.597490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:27.631241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] MFA Rules not processed for user `6b3a0451e20f42a38ba6dc524a097363`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:27.632552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:27.773470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 929/1854] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:27.777448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:27.803359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a231-f915-4d91-8df1-bc0c0b8ae2d0 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:27.816937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:27.818824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:27.821480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:27.822840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:27.825437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:27.833896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:27.934988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 926/1855] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:27 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:27.935607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:27.966940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32314be4-e4ad-4e3d-9253-553ee7ac0a57 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:28.148569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:28.152628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:28.152628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:28.152628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:28.152628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:28.165398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:28.210418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 930/1856] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:27 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:28.216287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:28.231606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252f2b9-5ee0-4714-b5f1-74d567d55ede admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:28.237810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:28.239045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:28.239493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:28.239777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:28.240524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:28.242142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:28.256733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 927/1857] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:28.261724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:28.276096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3de49-8002-4001-8274-49b6526bb471 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:28.281789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:28.282985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:28.283469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:28.283929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] PATH_INFO: `/v3/projects/12b26dcb1b284e9d9a736596006e3507/users/96d92f5c48d04582ba3478ffe2188e8d/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:28.311248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b26dcb1b284e9d9a736596006e3507, user_id=96d92f5c48d04582ba3478ffe2188e8d, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:28.314286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:28.340505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 931/1858] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:28 2022] PUT /identity/v3/projects/12b26dcb1b284e9d9a736596006e3507/users/96d92f5c48d04582ba3478ffe2188e8d/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:28.344525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:28.355296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d41f1-8424-48bf-9c47-788f25f7ac4b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:28.432562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c15a3-d24b-4bf5-9e91-e9eb7842a1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:28.433260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c15a3-d24b-4bf5-9e91-e9eb7842a1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:28.433804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c15a3-d24b-4bf5-9e91-e9eb7842a1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:28.434095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c15a3-d24b-4bf5-9e91-e9eb7842a1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:28.434845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c15a3-d24b-4bf5-9e91-e9eb7842a1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:28.437146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c15a3-d24b-4bf5-9e91-e9eb7842a1f3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:28.452075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 928/1859] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:28.456921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:28.474243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887fc34-2320-455f-a527-c049af00829d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:28.477029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:28.478236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:28.478621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:28.479003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] PATH_INFO: `/v3/projects/12b26dcb1b284e9d9a736596006e3507/users/96d92f5c48d04582ba3478ffe2188e8d/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:28.481267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b26dcb1b284e9d9a736596006e3507, user_id=96d92f5c48d04582ba3478ffe2188e8d, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:28.484568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:28.499975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 932/1860] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:28 2022] PUT /identity/v3/projects/12b26dcb1b284e9d9a736596006e3507/users/96d92f5c48d04582ba3478ffe2188e8d/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:30.113088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:30.113562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:30.113867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:30.181593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] MFA Rules not processed for user `96d92f5c48d04582ba3478ffe2188e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:30.182906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:30.224589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:30.276567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 929/1861] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:30.295153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:30.298006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:30.368054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:30.422309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40511b2-9788-46bf-bb72-ecec34e56104 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:30.437989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d238504-5510-4816-b88b-df631c3a4c9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:30.440055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d238504-5510-4816-b88b-df631c3a4c9c service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:30.440834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d238504-5510-4816-b88b-df631c3a4c9c service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:30.441201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d238504-5510-4816-b88b-df631c3a4c9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:30.612204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d238504-5510-4816-b88b-df631c3a4c9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:30.613896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d238504-5510-4816-b88b-df631c3a4c9c service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:30.645251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 933/1862] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:53:31.572636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:31.574305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:31.710035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc8fa-9e39-4ed3-b5c3-6313c0704406 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:31.717413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:31.718300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:31.718540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:31.718821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] PATH_INFO: `/v3/users/8115ef190ebb45d89b28ef235aff2e59` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:31.720092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=8115ef190ebb45d89b28ef235aff2e59)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:31.722249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:31.765799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6dd3-26db-418a-a789-3edd983693fa None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:31.766265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6dd3-26db-418a-a789-3edd983693fa None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:31.767103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a6dd3-26db-418a-a789-3edd983693fa None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:31.769668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 934/1863] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:53:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:53:31.875666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8115ef190ebb45d89b28ef235aff2e59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:31.920955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:31.921256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:31.921546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:31.925212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8115ef190ebb45d89b28ef235aff2e59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:31.942821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8115ef190ebb45d89b28ef235aff2e59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:31.964908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8115ef190ebb45d89b28ef235aff2e59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:31.965708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-8bfb1b07-6f80-4d47-9bf4-0ada91a530ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8115ef190ebb45d89b28ef235aff2e59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:31.985247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 930/1864] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:31 2022] DELETE /identity/v3/users/8115ef190ebb45d89b28ef235aff2e59 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:31.993093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:31.999102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:31.999588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:32.024335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:32.025596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:32.063714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:32.065108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:32.185975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 935/1865] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:53:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:32.188066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:32.230353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 931/1866] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:32.328915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:32.331317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:32.416931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800d37-ca4f-4d91-8e62-abcc69394e45 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:32.421078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:32.422371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:32.422803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:32.423285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:32.436319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:32.437783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:32.440239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:32.443793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:32.467994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 936/1867] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:32.588701 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799023b9-f483-40a8-ae68-37a900d9f079 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:32.594026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:32.595335 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:32.596763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:32.597065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:32.599706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:32.705532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:32.707085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b0432e-6ee9-4547-9405-364a67a075fd service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:32.749607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 932/1868] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:33.354796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:33.357189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:33.377647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09016f0c-be23-4670-b443-7517509dc78b service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:33.386373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:33.387731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.388006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.388785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.390925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401ae95-1c94-4d88-8dab-6b346faa18b8 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.391280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401ae95-1c94-4d88-8dab-6b346faa18b8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.391586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401ae95-1c94-4d88-8dab-6b346faa18b8 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.393348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 933/1869] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:53:33.403621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:33.404098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.404555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.405004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.405311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:33.429169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 937/1870] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:33.463195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:33.466360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:33.492090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 934/1871] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:53:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:33.524845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:33.538478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ad3c9-dd8f-45fb-870d-31ea3d83609a service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:33.545040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f022387e-07f6-45be-85d7-6138329fdb43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:33.545786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022387e-07f6-45be-85d7-6138329fdb43 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.546734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022387e-07f6-45be-85d7-6138329fdb43 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.547178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f022387e-07f6-45be-85d7-6138329fdb43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.554052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:33.558050 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022387e-07f6-45be-85d7-6138329fdb43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:33.559663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f022387e-07f6-45be-85d7-6138329fdb43 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:33.577583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 938/1872] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:33.656741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:33.658195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:33.674636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb1eb8-5643-4946-b4d7-7794201044a4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:33.683798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:33.684690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.684979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.685257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] PATH_INFO: `/v3/projects/762fed7c2947439897af16bd2d6b84db` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.686504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762fed7c2947439897af16bd2d6b84db)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:33.688477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:33.748064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762fed7c2947439897af16bd2d6b84db', 'request_id': 'req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:33.759551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762fed7c2947439897af16bd2d6b84db', 'request_id': 'req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:33.765976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762fed7c2947439897af16bd2d6b84db', 'request_id': 'req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:33.783700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] The token cache is being invalidate because project 762fed7c2947439897af16bd2d6b84db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:33.783999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:33.784676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-399ecfbf-ed83-46ef-8a5e-3aa89eaff4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:33.791151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 935/1873] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:33 2022] DELETE /identity/v3/projects/762fed7c2947439897af16bd2d6b84db => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:33.793653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.793944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.794219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.816877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:33.819049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:33.904674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 939/1874] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:33.912273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.913480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.913480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.970020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:33.970020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:33.970020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 936/1875] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:33.972773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:33.986214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36877c-5b72-4ab0-af6e-82d20c777a80 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:33.992761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:33.993786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:33.994055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:33.994767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:33.995641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:33.996245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:33.997935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.023276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 940/1876] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:34.027625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.032616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:34.045469 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e61c16-a251-41e4-915c-5ce74032c555 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:34.054944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:34.055962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.056409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.056837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.057892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:34.060212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.112244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 937/1877] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:34 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:53:34.118458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.137464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437521e-5b35-4f05-affa-63c6a7dd57d6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:34.238306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:34.240610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.240961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.241370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.242293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:34.246532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.299334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 941/1878] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:34 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:34.300668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.323647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285efd7a-0164-47ee-af9f-94bfe1dce87c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:34.333434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:34.334511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.335381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.335681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.336457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:34.338399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.372357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 938/1879] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:34.376545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.400594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b8a5f-b580-4489-b899-5f016222190b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:34.415826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f492e2-ab11-46bc-91e5-2ec853a79e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:34.417034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f492e2-ab11-46bc-91e5-2ec853a79e68 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.417731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f492e2-ab11-46bc-91e5-2ec853a79e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.419386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f492e2-ab11-46bc-91e5-2ec853a79e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.420354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f492e2-ab11-46bc-91e5-2ec853a79e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:34.424095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f492e2-ab11-46bc-91e5-2ec853a79e68 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.444459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 942/1880] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:34.447789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.468844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dcfc-982f-4c57-9b02-47ebdb783ccf admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:34.475597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:34.476527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.476808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.477088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] PATH_INFO: `/v3/projects/e1a4973f3ab54b0cb9afdbcef21e1ee1/users/dc8fd271edae4247a829c7f196c45c2f/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.507446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a4973f3ab54b0cb9afdbcef21e1ee1, user_id=dc8fd271edae4247a829c7f196c45c2f, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:34.510184 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.555580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 939/1881] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:34 2022] PUT /identity/v3/projects/e1a4973f3ab54b0cb9afdbcef21e1ee1/users/dc8fd271edae4247a829c7f196c45c2f/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:34.560526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.561023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.561490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.608136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] MFA Rules not processed for user `dc8fd271edae4247a829c7f196c45c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:34.610150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:34.686223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:34.737388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 943/1882] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:34.742434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.864742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a433e5-753b-4674-96db-d5cacdfd041d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:34.870601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:34.872045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:34.872404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:34.872907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:34.873944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:34.876082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:34.918670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 940/1883] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:34 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:34.919887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:34.933666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb1f08-41d3-4fdd-96d7-56d33a1b7740 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.006603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.007647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.008139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.008594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.009445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.011650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.036277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 944/1884] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:34 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:35.039795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.052051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de0056-a0b1-4f14-b3c9-8066bf2e1ecc admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.058218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.058909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.059281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.059720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.060879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.063119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.075979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 941/1885] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:35.080373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.097776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397ff00-01b5-4015-81e5-431df939eb45 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.105712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0e3fe-44ab-4b29-a0d7-ab269f557155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.106417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0e3fe-44ab-4b29-a0d7-ab269f557155 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.106703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0e3fe-44ab-4b29-a0d7-ab269f557155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.106991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0e3fe-44ab-4b29-a0d7-ab269f557155 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.107909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0e3fe-44ab-4b29-a0d7-ab269f557155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.109574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0e3fe-44ab-4b29-a0d7-ab269f557155 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.123356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 945/1886] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:35.127359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.139336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e680f-4a3f-48ba-af4c-80f11641fd81 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.144240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.145237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.146300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.146649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] PATH_INFO: `/v3/projects/f4bdb99281f24257bff18abd7cfb8c47/users/7bdf6453ee554d18a551ae8154f1ae8d/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.170714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4bdb99281f24257bff18abd7cfb8c47, user_id=7bdf6453ee554d18a551ae8154f1ae8d, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.173549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.189099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 942/1887] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:35 2022] PUT /identity/v3/projects/f4bdb99281f24257bff18abd7cfb8c47/users/7bdf6453ee554d18a551ae8154f1ae8d/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:35.199844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789299a-d599-411c-8667-b84f20993b8b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.200336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789299a-d599-411c-8667-b84f20993b8b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.200743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789299a-d599-411c-8667-b84f20993b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.243757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9789299a-d599-411c-8667-b84f20993b8b None None] MFA Rules not processed for user `7bdf6453ee554d18a551ae8154f1ae8d`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:35.245630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9789299a-d599-411c-8667-b84f20993b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:35.332280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 946/1888] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:35.337189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.407203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:35.414234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce729b5-fcea-4de1-b0c4-b26696f29339 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.420126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.420800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.421080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.421355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.422094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.424206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.451924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 943/1889] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:35 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:35.456554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789299a-d599-411c-8667-b84f20993b8b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.468421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789299a-d599-411c-8667-b84f20993b8b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.535820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:35.544428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.545371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.546247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.546247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.546770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.548949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.578208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 947/1890] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:35 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:35.579459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.594030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2eecd4-0ce0-4476-97c4-859714895c9e admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.599288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.599990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.600251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.600613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.601368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.603118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.619036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 944/1891] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:35.620577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.652581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde90cc0-c8da-4e0e-9174-8d2adac436f1 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.671260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.672840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.673391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.673773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] PATH_INFO: `/v3/projects/2c5b96d8ea894b489bcbe1dc890506ac/users/25abc61a4e924237811897ea7c2db765/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.692867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5b96d8ea894b489bcbe1dc890506ac, user_id=25abc61a4e924237811897ea7c2db765, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.695998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.716976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 948/1892] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:35 2022] PUT /identity/v3/projects/2c5b96d8ea894b489bcbe1dc890506ac/users/25abc61a4e924237811897ea7c2db765/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:35.721028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.733593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5baf61-e36e-4ba2-883a-233a565fba6a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.822443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcc28f-9ff3-455a-b9b7-f1880528f694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.824038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc28f-9ff3-455a-b9b7-f1880528f694 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.824479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc28f-9ff3-455a-b9b7-f1880528f694 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.824905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc28f-9ff3-455a-b9b7-f1880528f694 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.825817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcc28f-9ff3-455a-b9b7-f1880528f694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.827659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcc28f-9ff3-455a-b9b7-f1880528f694 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.843448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 945/1893] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:35.845681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:35.858899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443110d-c482-4750-a930-67a3cc582a4b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:35.866731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:35.867649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.868032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.868203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] PATH_INFO: `/v3/projects/2c5b96d8ea894b489bcbe1dc890506ac/users/25abc61a4e924237811897ea7c2db765/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.871588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5b96d8ea894b489bcbe1dc890506ac, user_id=25abc61a4e924237811897ea7c2db765, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:35.875175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:35.888607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 949/1894] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:35 2022] PUT /identity/v3/projects/2c5b96d8ea894b489bcbe1dc890506ac/users/25abc61a4e924237811897ea7c2db765/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:35.899592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:35.900002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:35.900893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:35.935688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] MFA Rules not processed for user `25abc61a4e924237811897ea7c2db765`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:35.945461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:36.071348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 946/1895] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:36.083405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:36.085154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:36.145694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbb7e5-5dd5-4a63-bdee-5ae3f4d00109 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:36.149496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:36.150153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:36.150438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:36.150722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:36.217608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:36.219160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:36.229500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 950/1896] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:37.845544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:37.856785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68148d1-dfae-4f54-99e9-75300317b24b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:37.860411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:37.861075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:37.861359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:37.861634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:37.922108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:37.935414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:37.936992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:37.951349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 947/1897] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:38.611044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:38.612486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:38.625233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0cacd-b77e-4f30-a42c-50c64ec83d67 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:38.631140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0449c-112b-4013-9024-2073a6110a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:38.631719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0449c-112b-4013-9024-2073a6110a2a service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:38.631999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0449c-112b-4013-9024-2073a6110a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:38.632274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0449c-112b-4013-9024-2073a6110a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:38.646395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0449c-112b-4013-9024-2073a6110a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:38.648324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0449c-112b-4013-9024-2073a6110a2a service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:38.659402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 951/1898] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:42.959875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:43.073067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0af92f-6905-41f1-b70e-21a5586313d1 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:43.079467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:43.079947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:43.080241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:43.080522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] PATH_INFO: `/v3/users/dc8fd271edae4247a829c7f196c45c2f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:43.082270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8fd271edae4247a829c7f196c45c2f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:43.089012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:43.157686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8fd271edae4247a829c7f196c45c2f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.174706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8fd271edae4247a829c7f196c45c2f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.187484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8fd271edae4247a829c7f196c45c2f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.195941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8fd271edae4247a829c7f196c45c2f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.196632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8fd271edae4247a829c7f196c45c2f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.211514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 948/1899] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:42 2022] DELETE /identity/v3/users/dc8fd271edae4247a829c7f196c45c2f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:43.216073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:43.216556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:43.216691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:43.235521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:43.236774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:43.290923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:43.306241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 952/1900] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:43.315263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:43.316739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:43.378856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161c77c-8e87-4fd7-b6b6-81ac4bd37364 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:43.382722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:43.383408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:43.383697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:43.383978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:43.394037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:43.395715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:43.405950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 949/1901] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:43.771590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:43.773200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:43.791383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674a3e2-b027-41e1-9183-fdcf7ec097c3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:43.798884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:43.799810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:43.800088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:43.800358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] PATH_INFO: `/v3/projects/e1a4973f3ab54b0cb9afdbcef21e1ee1` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:43.801733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a4973f3ab54b0cb9afdbcef21e1ee1)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:43.806391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:43.884859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a4973f3ab54b0cb9afdbcef21e1ee1', 'request_id': 'req-dd148dc1-3915-4876-8595-476187111838'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.894124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a4973f3ab54b0cb9afdbcef21e1ee1', 'request_id': 'req-dd148dc1-3915-4876-8595-476187111838'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.903079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a4973f3ab54b0cb9afdbcef21e1ee1', 'request_id': 'req-dd148dc1-3915-4876-8595-476187111838'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.921890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] The token cache is being invalidate because project e1a4973f3ab54b0cb9afdbcef21e1ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:43.922173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.923045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:43.925330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 953/1902] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:43 2022] DELETE /identity/v3/projects/e1a4973f3ab54b0cb9afdbcef21e1ee1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:53:43.931405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:43.943287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cd1c9-9c8d-4b85-94dd-fd7848181f0d service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:43.955601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:44.023305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:44.024197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:44.024485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:44.024763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] PATH_INFO: `/v3/users/7bdf6453ee554d18a551ae8154f1ae8d` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:44.026054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bdf6453ee554d18a551ae8154f1ae8d)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:44.028039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:44.084790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bdf6453ee554d18a551ae8154f1ae8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.103258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bdf6453ee554d18a551ae8154f1ae8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.111002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bdf6453ee554d18a551ae8154f1ae8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.119636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bdf6453ee554d18a551ae8154f1ae8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.120589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bdf6453ee554d18a551ae8154f1ae8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.133772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 950/1903] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:43 2022] DELETE /identity/v3/users/7bdf6453ee554d18a551ae8154f1ae8d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:44.613634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:44.617474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:44.640484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd148dc1-3915-4876-8595-476187111838 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:44.704408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:44.705193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:44.705584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:44.705759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] PATH_INFO: `/v3/projects/f4bdb99281f24257bff18abd7cfb8c47` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:44.708200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4bdb99281f24257bff18abd7cfb8c47)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:44.709285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:44.756482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4bdb99281f24257bff18abd7cfb8c47', 'request_id': 'req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.762643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4bdb99281f24257bff18abd7cfb8c47', 'request_id': 'req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.768260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4bdb99281f24257bff18abd7cfb8c47', 'request_id': 'req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.783499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] The token cache is being invalidate because project f4bdb99281f24257bff18abd7cfb8c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:44.783823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.784473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.786638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 954/1904] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:44 2022] DELETE /identity/v3/projects/f4bdb99281f24257bff18abd7cfb8c47 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:44.790816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:44.792268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:44.806688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224b8bbb-59b3-4bc3-b547-c98bde4c6ca9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:44.866348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:44.867258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:44.867539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:44.867814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] PATH_INFO: `/v3/users/25abc61a4e924237811897ea7c2db765` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:44.869273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=25abc61a4e924237811897ea7c2db765)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:44.871302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:44.914215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:44.931156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25abc61a4e924237811897ea7c2db765'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.945146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25abc61a4e924237811897ea7c2db765'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.952434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25abc61a4e924237811897ea7c2db765'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.960324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25abc61a4e924237811897ea7c2db765'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.960967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ec5604b5-1a35-4d91-9b3a-6b25029c2c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25abc61a4e924237811897ea7c2db765'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:44.968247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 951/1905] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:44 2022] DELETE /identity/v3/users/25abc61a4e924237811897ea7c2db765 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:45.384814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:45.386264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:45.396894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:45.410427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7bbd3b-52f9-41b4-ae04-f7f74163ea20 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:45.479581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:45.480680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.480966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.481274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] PATH_INFO: `/v3/projects/2c5b96d8ea894b489bcbe1dc890506ac` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.482675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5b96d8ea894b489bcbe1dc890506ac)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:45.484568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:45.533087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5b96d8ea894b489bcbe1dc890506ac', 'request_id': 'req-9de34c4c-be72-4d86-8a92-a3cef5579eb4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:45.539494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5b96d8ea894b489bcbe1dc890506ac', 'request_id': 'req-9de34c4c-be72-4d86-8a92-a3cef5579eb4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:45.545139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5b96d8ea894b489bcbe1dc890506ac', 'request_id': 'req-9de34c4c-be72-4d86-8a92-a3cef5579eb4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:45.558822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] The token cache is being invalidate because project 2c5b96d8ea894b489bcbe1dc890506ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:45.559141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:45.559773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9de34c4c-be72-4d86-8a92-a3cef5579eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:45.561336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 955/1906] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:45 2022] DELETE /identity/v3/projects/2c5b96d8ea894b489bcbe1dc890506ac => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:45.566499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.567503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.567733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.588171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:45.589437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:45.671237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 952/1907] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:45.679857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.680432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.680432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.700613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:45.702093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:45.711870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 956/1908] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:45.716382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:45.727681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ab5d3-b6f2-4819-8b5b-8d3f6387a324 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:45.733164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:45.733946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.734127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.734408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.735215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:45.735525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:45.737169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:45.755128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 953/1909] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:45.759587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:45.769559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa704152-ea5b-4933-ada4-9e0a8a666619 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:45.776291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:45.776990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.777274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.777552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.778265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:45.780224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:45.814264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 957/1910] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:45 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:53:45.819404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:45.835683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552632b-3fcc-4a4c-972d-de65b046b3b7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:45.905610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:45.906756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.906756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.907025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.908159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:45.911865 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:45.950806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:45.951295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 954/1911] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:45 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:45.967822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d24ced-67bd-4bd8-8ae0-94a5f94658b7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:45.974517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:45.975249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:45.975546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:45.975822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:45.976658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:45.978361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:45.991740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 958/1912] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:45.996436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:46.008948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da467fb8-2486-4f33-b0f2-8b42c2c31e81 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:46.014266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a00e45-e39a-469e-ac1b-666b34c71c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:46.014940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a00e45-e39a-469e-ac1b-666b34c71c78 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:46.015252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a00e45-e39a-469e-ac1b-666b34c71c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:46.015525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a00e45-e39a-469e-ac1b-666b34c71c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:46.016261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a00e45-e39a-469e-ac1b-666b34c71c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:46.017860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a00e45-e39a-469e-ac1b-666b34c71c78 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:46.031836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 955/1913] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:53:46.034691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:46.045937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823085c-a985-48f9-970e-8eed7e43ed3c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:46.050784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:46.052298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:46.052680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:46.052964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] PATH_INFO: `/v3/projects/9a43767d64494137af74bbb2955ef55a/users/9c901aba30d141b39e7189eefc7c1ce1/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:46.072033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a43767d64494137af74bbb2955ef55a, user_id=9c901aba30d141b39e7189eefc7c1ce1, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:46.075148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:46.096534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 959/1914] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:46 2022] PUT /identity/v3/projects/9a43767d64494137af74bbb2955ef55a/users/9c901aba30d141b39e7189eefc7c1ce1/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:46.102109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c64b0-8875-411a-8a5a-2490cada94c4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:46.102480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c64b0-8875-411a-8a5a-2490cada94c4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:46.102764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c64b0-8875-411a-8a5a-2490cada94c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:46.125458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-bc0c64b0-8875-411a-8a5a-2490cada94c4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:46.127212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bc0c64b0-8875-411a-8a5a-2490cada94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:46.139249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:46.198965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 956/1915] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:53:46.210791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:46.212634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:46.248727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:46.279851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dc06b-5db5-4ef8-98a6-b134f612d367 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:46.283759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:46.284477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:46.284843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:46.285183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:46.298290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:46.299972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:46.313992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 960/1916] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:47.196246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa47f1-c87a-460a-8982-3c5d69c940a6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:47.196812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa47f1-c87a-460a-8982-3c5d69c940a6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:47.196812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa47f1-c87a-460a-8982-3c5d69c940a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:47.229561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-00aa47f1-c87a-460a-8982-3c5d69c940a6 None None] MFA Rules not processed for user `9c901aba30d141b39e7189eefc7c1ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:47.230818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-00aa47f1-c87a-460a-8982-3c5d69c940a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:47.332295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 957/1917] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:47.344372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:47.345926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:47.423700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da05e3-2736-4bc2-87b9-f5e1d6da8923 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:47.428017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646e28a-ac2a-4565-823e-e2a425e6d073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:47.428675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e28a-ac2a-4565-823e-e2a425e6d073 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:47.428960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e28a-ac2a-4565-823e-e2a425e6d073 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:47.429236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646e28a-ac2a-4565-823e-e2a425e6d073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:47.438234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646e28a-ac2a-4565-823e-e2a425e6d073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:47.439726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646e28a-ac2a-4565-823e-e2a425e6d073 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:47.453690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 961/1918] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:50.073286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f5aff-c365-435f-800c-2a23517bc2b9 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:50.073786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f5aff-c365-435f-800c-2a23517bc2b9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:50.073885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f5aff-c365-435f-800c-2a23517bc2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:50.096403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c19f5aff-c365-435f-800c-2a23517bc2b9 None None] MFA Rules not processed for user `9c901aba30d141b39e7189eefc7c1ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:50.097639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c19f5aff-c365-435f-800c-2a23517bc2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:50.107232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 958/1919] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:51.028842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fcc81-8c1b-421e-9602-9c074165f72e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:51.029371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fcc81-8c1b-421e-9602-9c074165f72e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:51.029489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8fcc81-8c1b-421e-9602-9c074165f72e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:51.030784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 962/1920] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:53:51.076268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:51.077919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:51.078807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:51.122501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:51.132533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:51.273017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 959/1921] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:53:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:51.378691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fcc81-8c1b-421e-9602-9c074165f72e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:51.381948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4e8fcc81-8c1b-421e-9602-9c074165f72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:51.438736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8fcc81-8c1b-421e-9602-9c074165f72e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:51.451761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:51.453727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:51.454044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:51.454334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:51.482029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:51.487583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:51.503282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 963/1922] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:53:51.961950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:51.968145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:51.985733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652fc6b-4173-4729-ad5b-b541f3e484b3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:51.994107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:51.994790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:51.995290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:51.995743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] PATH_INFO: `/v3/users/6b3a0451e20f42a38ba6dc524a097363` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:51.997228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3a0451e20f42a38ba6dc524a097363)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:51.999254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:52.040825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:52.058545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3a0451e20f42a38ba6dc524a097363'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.072409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3a0451e20f42a38ba6dc524a097363'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.079651 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3a0451e20f42a38ba6dc524a097363'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.088508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3a0451e20f42a38ba6dc524a097363'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.089117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3a0451e20f42a38ba6dc524a097363'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.095725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 960/1923] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:51 2022] DELETE /identity/v3/users/6b3a0451e20f42a38ba6dc524a097363 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:52.283720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:52.297729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7604399-042f-4d63-89c0-54c122ac76ee service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:52.375858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:52.376736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:52.377199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:52.377659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] PATH_INFO: `/v3/projects/2b546ac5537046a3a9b17bb33915f440` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:52.378909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b546ac5537046a3a9b17bb33915f440)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:52.381256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:52.473026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b546ac5537046a3a9b17bb33915f440', 'request_id': 'req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.484250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b546ac5537046a3a9b17bb33915f440', 'request_id': 'req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.493842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b546ac5537046a3a9b17bb33915f440', 'request_id': 'req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.499960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:52.516462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] The token cache is being invalidate because project 2b546ac5537046a3a9b17bb33915f440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:52.516788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.517422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.519298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 964/1924] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:52 2022] DELETE /identity/v3/projects/2b546ac5537046a3a9b17bb33915f440 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:52.524437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:52.525986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:52.541402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b9699-90d7-47c9-abbb-5720909fb980 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:52.624190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:52.625527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:52.626037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:52.626511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] PATH_INFO: `/v3/users/96d92f5c48d04582ba3478ffe2188e8d` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:52.628540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d92f5c48d04582ba3478ffe2188e8d)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:52.632423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:52.694632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d92f5c48d04582ba3478ffe2188e8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.711600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d92f5c48d04582ba3478ffe2188e8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.718934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d92f5c48d04582ba3478ffe2188e8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.727722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d92f5c48d04582ba3478ffe2188e8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.730080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b14a841-7344-48a9-a5bc-116117312020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d92f5c48d04582ba3478ffe2188e8d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:52.737907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 961/1925] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:53:52 2022] DELETE /identity/v3/users/96d92f5c48d04582ba3478ffe2188e8d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:53.390184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:53.392957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:53.409228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde2ab0-24de-4f9a-a129-f61a0c14baf4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:53.499023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:53.500341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:53.500632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:53.500893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] PATH_INFO: `/v3/projects/12b26dcb1b284e9d9a736596006e3507` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:53.502749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b26dcb1b284e9d9a736596006e3507)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:53.506126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:53.578937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b26dcb1b284e9d9a736596006e3507', 'request_id': 'req-037daccc-2aff-4ad2-b863-26afe986b7f7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:53.586395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b26dcb1b284e9d9a736596006e3507', 'request_id': 'req-037daccc-2aff-4ad2-b863-26afe986b7f7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:53.593484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b26dcb1b284e9d9a736596006e3507', 'request_id': 'req-037daccc-2aff-4ad2-b863-26afe986b7f7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:53.612228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] The token cache is being invalidate because project 12b26dcb1b284e9d9a736596006e3507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:53:53.612590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:53.613169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-037daccc-2aff-4ad2-b863-26afe986b7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:53:53.615595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 965/1926] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:53:53 2022] DELETE /identity/v3/projects/12b26dcb1b284e9d9a736596006e3507 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:53.622619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:53.623199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:53.624278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:53.654591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:53.656036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:53.698000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:53.784481 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 962/1927] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:53.787046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:53.787680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:53.788153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:53.828241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:53.829934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:53.840699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 966/1928] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:53.845516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:53.862973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b42c80-73f6-42af-8d95-f9a0c84f6c44 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:53.869994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:53.871495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:53.871994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:53.872477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:53.873690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:53:53.874525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:53.876468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:53.897810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 963/1929] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:53:53.898820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:53.911459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcf406-9677-4435-ae49-02ac2e8df233 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:53.918951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:53.919884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:53.920227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:53.920505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:53.921216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:53.923253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:53.959114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 967/1930] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:53:53 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:53.960343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:53.972973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad798ed-1286-47b5-9ca7-11d027451eb1 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:54.061122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:54.062247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:54.062709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:54.063189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:54.064262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:54.066710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:54.097299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 964/1931] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:53:53 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:53:54.099860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:54.115575 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5a2a2-ad02-455f-a78a-932a01c661f2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:54.122272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:54.123396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:54.123902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:54.124375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:54.125374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:54.127462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:54.143605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 968/1932] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:54.150755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:54.164517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2bf05d-21cb-406b-90c5-3b73526d90eb admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:54.173697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7530706-d1b5-4238-bf13-e03364754e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:54.174389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530706-d1b5-4238-bf13-e03364754e47 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:54.174668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530706-d1b5-4238-bf13-e03364754e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:54.175721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7530706-d1b5-4238-bf13-e03364754e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:54.175841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7530706-d1b5-4238-bf13-e03364754e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:54.178541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7530706-d1b5-4238-bf13-e03364754e47 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:54.196852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 965/1933] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:53:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:53:54.198116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:54.210028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f2c3f-8578-47d7-849b-31f8bf9a602b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:54.216983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:54.217950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:54.218234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:54.219079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] PATH_INFO: `/v3/projects/48b54332d600458e9a989ed07ee8e4b2/users/c7563df960b54fbdbe083b60f91971a6/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:54.238317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b54332d600458e9a989ed07ee8e4b2, user_id=c7563df960b54fbdbe083b60f91971a6, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:54.241797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:54.274806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 969/1934] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:53:54 2022] PUT /identity/v3/projects/48b54332d600458e9a989ed07ee8e4b2/users/c7563df960b54fbdbe083b60f91971a6/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:53:54.285985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:54.285985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:54.285985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:54.327851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] MFA Rules not processed for user `c7563df960b54fbdbe083b60f91971a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:53:54.329307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:54.432932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 966/1935] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:53:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:53:54.447401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:53:54.450344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:53:54.516179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:53:54.645407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8157d35d-e68c-41ad-b13a-1aa2885d05ce admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:53:54.650149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639272cf-5ace-48b4-8f27-d99eb98c2bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:53:54.651504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639272cf-5ace-48b4-8f27-d99eb98c2bfa service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:53:54.652017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639272cf-5ace-48b4-8f27-d99eb98c2bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:53:54.652501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639272cf-5ace-48b4-8f27-d99eb98c2bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:53:54.669496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639272cf-5ace-48b4-8f27-d99eb98c2bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:53:54.671187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639272cf-5ace-48b4-8f27-d99eb98c2bfa service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:53:54.682578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 970/1936] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:53:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:04.429536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:04.677829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=17 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:12.104874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:12.106416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:12.193202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027f003-3274-49c9-a28c-29ad8440624c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:12.198373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:12.199305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:12.200314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:12.200640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] PATH_INFO: `/v3/users/364d8579a2484c75ba039fc319f86ac7` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:12.201926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=364d8579a2484c75ba039fc319f86ac7)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:12.203952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:12.273952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364d8579a2484c75ba039fc319f86ac7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:12.290589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364d8579a2484c75ba039fc319f86ac7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:12.297890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364d8579a2484c75ba039fc319f86ac7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:12.307502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364d8579a2484c75ba039fc319f86ac7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:12.307502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364d8579a2484c75ba039fc319f86ac7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:12.314507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 967/1937] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:12 2022] DELETE /identity/v3/users/364d8579a2484c75ba039fc319f86ac7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:12.319911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:12.320460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:12.320460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:12.338029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:12.339282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:12.417259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 971/1938] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:12.428235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:12.429710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:12.497804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16045f07-243a-4fae-af33-f74e20e256c9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:12.501907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:12.502688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:12.503085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:12.503480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:12.514658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:12.516394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:12.527205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 968/1939] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:12.891621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:12.893132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:12.907023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba97246-e521-4228-8038-79d9d4de4d28 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:12.920596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:12.921493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:12.921883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:12.922029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] PATH_INFO: `/v3/projects/d6a16b01609947638fea7376866fe19c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:12.924345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a16b01609947638fea7376866fe19c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:12.926219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:12.989853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a16b01609947638fea7376866fe19c', 'request_id': 'req-8c8e2b4b-5345-44f2-a463-deb285d34fac'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.000315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a16b01609947638fea7376866fe19c', 'request_id': 'req-8c8e2b4b-5345-44f2-a463-deb285d34fac'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.006602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a16b01609947638fea7376866fe19c', 'request_id': 'req-8c8e2b4b-5345-44f2-a463-deb285d34fac'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.022423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] The token cache is being invalidate because project d6a16b01609947638fea7376866fe19c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:13.022709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.023902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.025530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 972/1940] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:12 2022] DELETE /identity/v3/projects/d6a16b01609947638fea7376866fe19c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:13.029581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:13.040589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f72b51c-c02d-4084-af84-5fea37d7d660 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:13.064769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:13.141087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:13.142402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:13.142402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:13.142631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] PATH_INFO: `/v3/users/efbd775a098240a1b2773c5ed700caad` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:13.144218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efbd775a098240a1b2773c5ed700caad)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:13.146907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:13.198808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efbd775a098240a1b2773c5ed700caad'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.212168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efbd775a098240a1b2773c5ed700caad'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.219181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efbd775a098240a1b2773c5ed700caad'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.227079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efbd775a098240a1b2773c5ed700caad'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.227845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1791a902-696e-4655-ba72-7db19f64d6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efbd775a098240a1b2773c5ed700caad'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.235514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 969/1941] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:13 2022] DELETE /identity/v3/users/efbd775a098240a1b2773c5ed700caad => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:13.594713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:13.596443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:13.620114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8e2b4b-5345-44f2-a463-deb285d34fac admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:13.707843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:13.709198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:13.709506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:13.709991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] PATH_INFO: `/v3/projects/8b88faf444ac472fb71fce8a890fa346` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:13.711400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b88faf444ac472fb71fce8a890fa346)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:13.715163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:13.757434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:13.782478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b88faf444ac472fb71fce8a890fa346', 'request_id': 'req-9c648a78-90d7-4191-9663-6b6367adc859'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.791561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b88faf444ac472fb71fce8a890fa346', 'request_id': 'req-9c648a78-90d7-4191-9663-6b6367adc859'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.798674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b88faf444ac472fb71fce8a890fa346', 'request_id': 'req-9c648a78-90d7-4191-9663-6b6367adc859'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.814494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] The token cache is being invalidate because project 8b88faf444ac472fb71fce8a890fa346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:13.814855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.815603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9c648a78-90d7-4191-9663-6b6367adc859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:13.818040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 973/1942] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:13 2022] DELETE /identity/v3/projects/8b88faf444ac472fb71fce8a890fa346 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:13.824268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:13.825962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:13.826278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:13.857527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:13.859272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:13.937965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 970/1943] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:13.946471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:13.947773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:13.948277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:13.981336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:13.982712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:13.995936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 974/1944] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:14.000231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.011158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4409db-e8ed-48c6-8e13-fb8dff10ac37 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.016921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.017685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.017971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.018380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.019250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:14.019566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.021394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.034614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 971/1945] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:54:14.039734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.051507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe87d3-92a4-4b36-bc67-83a29e685ed5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.057496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.058282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.058564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.058836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.059577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.061517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.095814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 975/1946] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:14 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:14.097618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.112033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4a89-c9a5-45df-bfc1-5e04ab09aca6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.149974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:14.186988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.187747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.188038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.188317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.189092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.191047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.216382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 972/1947] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:14 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:14.217647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.229105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7d309-a4f7-4ff3-b8fc-abc2bf9b9407 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.233818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.234499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.234778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.235152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.235867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.237498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.257175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 976/1948] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:14.258526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.275086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe83926-2e4a-4205-b171-07169b179971 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.281196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5267136-54d9-43da-bfaf-21ee6f8a7385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.281916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267136-54d9-43da-bfaf-21ee6f8a7385 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.282202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267136-54d9-43da-bfaf-21ee6f8a7385 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.282483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5267136-54d9-43da-bfaf-21ee6f8a7385 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.283262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267136-54d9-43da-bfaf-21ee6f8a7385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.284886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5267136-54d9-43da-bfaf-21ee6f8a7385 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.296763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 973/1949] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:14.301387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.312287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22897e-cb90-4866-886d-e72f3cae50db admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.317071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.318133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.318410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.318679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] PATH_INFO: `/v3/projects/b69d64390b2f4d38b24259b3c170ef47/users/4f5fb29e88ff47b28eb6390b7b31d7a6/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.342551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69d64390b2f4d38b24259b3c170ef47, user_id=4f5fb29e88ff47b28eb6390b7b31d7a6, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.345654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.361331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 977/1950] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:14 2022] PUT /identity/v3/projects/b69d64390b2f4d38b24259b3c170ef47/users/4f5fb29e88ff47b28eb6390b7b31d7a6/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:14.372812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.373104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.373389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.402704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] MFA Rules not processed for user `4f5fb29e88ff47b28eb6390b7b31d7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:14.403988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:14.477290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 974/1951] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:14.481792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.552671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6ef95-74d6-48c1-a2ce-41ba35b78369 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.557945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.558624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.558907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.559215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.559931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.561879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.592011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 978/1952] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:14 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:14.596391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.608043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938f02a-96ec-4d41-81a7-d372c423232b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.677864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.678627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.678908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.679228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.681448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.682185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.712096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 975/1953] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:14 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:14.713405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.726237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470be72-d1f7-441b-9a7e-d3e708ede355 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.731309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.731988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.732267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.732551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.733272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.734877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.751862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 979/1954] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:14.753581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.766570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb3512-933e-4020-82de-3d1bea457d12 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.771492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.772417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.772700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.772982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] PATH_INFO: `/v3/projects/a137e54520644281968f8c8b959d12da/users/ffcdcb7fbaf14e848c41acfd7641b060/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.794602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a137e54520644281968f8c8b959d12da, user_id=ffcdcb7fbaf14e848c41acfd7641b060, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.797361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.818365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 976/1955] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:14 2022] PUT /identity/v3/projects/a137e54520644281968f8c8b959d12da/users/ffcdcb7fbaf14e848c41acfd7641b060/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:14.822546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.833370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3984c7e-0699-419e-8074-e2c02f7e27cb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.856341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:14.898663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d3dca-9e69-40ab-9584-4121271f7ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.899382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d3dca-9e69-40ab-9584-4121271f7ffc admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.899667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d3dca-9e69-40ab-9584-4121271f7ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.899943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d3dca-9e69-40ab-9584-4121271f7ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.900678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d3dca-9e69-40ab-9584-4121271f7ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.902359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d3dca-9e69-40ab-9584-4121271f7ffc admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.914741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 980/1956] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:14.919276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:14.929276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:14.934349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a8824-5876-4974-948b-df47aac79502 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:14.946888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:14.948815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.949366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.950014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] PATH_INFO: `/v3/projects/a137e54520644281968f8c8b959d12da/users/ffcdcb7fbaf14e848c41acfd7641b060/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:14.953589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a137e54520644281968f8c8b959d12da, user_id=ffcdcb7fbaf14e848c41acfd7641b060, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:14.957813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:14.983150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 977/1957] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:14 2022] PUT /identity/v3/projects/a137e54520644281968f8c8b959d12da/users/ffcdcb7fbaf14e848c41acfd7641b060/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:14.987372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:14.987719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:14.988464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:15.030231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] MFA Rules not processed for user `ffcdcb7fbaf14e848c41acfd7641b060`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:15.032256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:15.143187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 981/1958] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:15.152734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:15.155806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:15.261525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbce69-0df3-4a67-9686-e6a23ad40cd8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:15.265451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:15.266116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:15.266401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:15.266682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:15.281309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:15.282863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:15.302841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 978/1959] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:15.404889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:15.416308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f82062-3804-4d18-ba0e-4236df45d14b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:15.420945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfbc55-9089-4e43-9589-1bbbafb98c9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:15.421980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfbc55-9089-4e43-9589-1bbbafb98c9b service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:15.422518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfbc55-9089-4e43-9589-1bbbafb98c9b service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:15.422883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfbc55-9089-4e43-9589-1bbbafb98c9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:15.506958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfbc55-9089-4e43-9589-1bbbafb98c9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:15.508495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfbc55-9089-4e43-9589-1bbbafb98c9b service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:15.527481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 982/1960] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:20.387295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:20.486524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a1d56-2fcf-4487-a023-330ea3141709 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:20.493073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:20.494488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:20.494488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:20.494764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] PATH_INFO: `/v3/users/4f5fb29e88ff47b28eb6390b7b31d7a6` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:20.496072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f5fb29e88ff47b28eb6390b7b31d7a6)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:20.498027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:20.572665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f5fb29e88ff47b28eb6390b7b31d7a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:20.592491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f5fb29e88ff47b28eb6390b7b31d7a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:20.605045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f5fb29e88ff47b28eb6390b7b31d7a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:20.616936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f5fb29e88ff47b28eb6390b7b31d7a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:20.619433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f5fb29e88ff47b28eb6390b7b31d7a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:20.628200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 979/1961] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:20 2022] DELETE /identity/v3/users/4f5fb29e88ff47b28eb6390b7b31d7a6 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:20.633378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:20.634139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:20.634139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:20.657432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:20.658856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:20.709845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:20.745442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 983/1962] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:20.753907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:20.756290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:20.778522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:20.862824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07de56c-85a9-4e4b-bfef-ee7e963650b7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:20.868643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:20.871384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:20.871659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:20.872083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:20.884686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:20.886770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:20.898949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 980/1963] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:21.412315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:21.414006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:21.427179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb27d1-aab7-4938-872f-641d91689749 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:21.432105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:21.433970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:21.433970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:21.433970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] PATH_INFO: `/v3/projects/b69d64390b2f4d38b24259b3c170ef47` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:21.435675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69d64390b2f4d38b24259b3c170ef47)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:21.438118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:21.512056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69d64390b2f4d38b24259b3c170ef47', 'request_id': 'req-3db69b65-49f5-4df4-97a0-89705b305196'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.519325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69d64390b2f4d38b24259b3c170ef47', 'request_id': 'req-3db69b65-49f5-4df4-97a0-89705b305196'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.529392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69d64390b2f4d38b24259b3c170ef47', 'request_id': 'req-3db69b65-49f5-4df4-97a0-89705b305196'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.548673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] The token cache is being invalidate because project b69d64390b2f4d38b24259b3c170ef47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:21.549241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.550340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.553418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 984/1964] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:21 2022] DELETE /identity/v3/projects/b69d64390b2f4d38b24259b3c170ef47 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:21.557553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:21.569398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c937bfd3-346d-4bcb-a008-8fb980023b35 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:21.640591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:21.641466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:21.641771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:21.642047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] PATH_INFO: `/v3/users/ffcdcb7fbaf14e848c41acfd7641b060` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:21.643351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffcdcb7fbaf14e848c41acfd7641b060)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:21.645936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:21.720652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffcdcb7fbaf14e848c41acfd7641b060'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.736164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffcdcb7fbaf14e848c41acfd7641b060'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.747197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffcdcb7fbaf14e848c41acfd7641b060'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.757254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffcdcb7fbaf14e848c41acfd7641b060'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.757934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7390f301-e8b6-4afa-8d72-e3ba797b9137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffcdcb7fbaf14e848c41acfd7641b060'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:21.771868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 981/1965] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:21 2022] DELETE /identity/v3/users/ffcdcb7fbaf14e848c41acfd7641b060 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:22.229628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.231227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:22.251103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db69b65-49f5-4df4-97a0-89705b305196 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.332285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.333978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.334185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.334364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] PATH_INFO: `/v3/projects/a137e54520644281968f8c8b959d12da` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.335649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a137e54520644281968f8c8b959d12da)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.337578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.386586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a137e54520644281968f8c8b959d12da', 'request_id': 'req-07ea1c27-47f7-4583-870d-4452a4015376'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:22.393329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a137e54520644281968f8c8b959d12da', 'request_id': 'req-07ea1c27-47f7-4583-870d-4452a4015376'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:22.399821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a137e54520644281968f8c8b959d12da', 'request_id': 'req-07ea1c27-47f7-4583-870d-4452a4015376'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:22.409267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:22.414607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] The token cache is being invalidate because project a137e54520644281968f8c8b959d12da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:22.415138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:22.416310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-07ea1c27-47f7-4583-870d-4452a4015376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:22.418002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 985/1966] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:22 2022] DELETE /identity/v3/projects/a137e54520644281968f8c8b959d12da => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:22.422888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.423279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.423592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.441987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:22.443776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:22.477834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:22.515932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 982/1967] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:22.524502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.524867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.525075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.547261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:22.548514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:22.558062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 986/1968] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:22.563319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.574367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112f725-f5de-412b-bed5-fd8d89e1d9d5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.579459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.580224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.580503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.580773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.581515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:22.581820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.583529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.602475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 983/1969] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:22.604609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.616922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbaaae-6561-4591-9cd4-3ae4ce0fa9b4 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.622169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.622849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.623164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.623443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.624193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.626137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.654618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 987/1970] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:22 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:22.659045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.670102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f947c8-59e9-4550-a1ea-e415069215f0 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.730678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.731395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.731672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.731943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.732636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.734575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.764222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 984/1971] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:22 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:22.768216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.779835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c5d71-6478-4730-9753-75c641ed9517 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.785361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.786150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.786402 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.786666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.787601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.789352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.804729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 988/1972] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:22.805939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.818339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c43cc-185f-4ed6-8dcc-cc3919e0b141 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.824142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a14a4-86e1-4a72-b80a-ee7062d74043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.824832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a14a4-86e1-4a72-b80a-ee7062d74043 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.825119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a14a4-86e1-4a72-b80a-ee7062d74043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.825399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a14a4-86e1-4a72-b80a-ee7062d74043 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.826140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a14a4-86e1-4a72-b80a-ee7062d74043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.827811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a14a4-86e1-4a72-b80a-ee7062d74043 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.839603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 985/1973] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:22.844030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:22.856001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd093-e6f9-4f95-817c-feeb69f42d2e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:22.861029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:22.861956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.862240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.862517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] PATH_INFO: `/v3/projects/0d1802ddd5094f44aeb6f56639aad876/users/010a59e6fa2f4953aed9bf726d1815c6/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.879354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d1802ddd5094f44aeb6f56639aad876, user_id=010a59e6fa2f4953aed9bf726d1815c6, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:22.882192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:22.895789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 989/1974] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:22 2022] PUT /identity/v3/projects/0d1802ddd5094f44aeb6f56639aad876/users/010a59e6fa2f4953aed9bf726d1815c6/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:22.907697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:22.907988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:22.908273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:22.938770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] MFA Rules not processed for user `010a59e6fa2f4953aed9bf726d1815c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:22.940161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:23.013831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 986/1975] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:23.018900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.104842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6da84f-af0b-45d8-8e1c-b5b93dae3af0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.110071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.110742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.111016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.111321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.112014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.113940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.144686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 990/1976] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:23 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:23.148972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.162319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4652a-09c8-4f18-a301-6623b732fe7b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.208075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:23.269119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.269890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.270697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.271049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.271827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.273880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.305018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 987/1977] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:23 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:23.310728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.346381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f199e34-6ba4-4ef1-a4fd-8592a96f7aa4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.369914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.371124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.371612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.372089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.373398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.376754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.407038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 991/1978] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:23.407970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.430673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7491-9271-42dd-bc46-fd1f9eb08bd9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.438153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.439741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.440492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.440947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] PATH_INFO: `/v3/projects/4e57a75632644a3ab09e7fcbc996b363/users/86d8ecde67484529afe6c02d7b0e6a5f/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.480630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e57a75632644a3ab09e7fcbc996b363, user_id=86d8ecde67484529afe6c02d7b0e6a5f, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.484210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.509766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 988/1979] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:23 2022] PUT /identity/v3/projects/4e57a75632644a3ab09e7fcbc996b363/users/86d8ecde67484529afe6c02d7b0e6a5f/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:23.514239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.530440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced477-6351-4416-90bc-89cf39995e4b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.615026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075522ad-9ba3-4341-91bc-ae1b26bab847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.615757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075522ad-9ba3-4341-91bc-ae1b26bab847 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.616038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075522ad-9ba3-4341-91bc-ae1b26bab847 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.616317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075522ad-9ba3-4341-91bc-ae1b26bab847 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.619447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075522ad-9ba3-4341-91bc-ae1b26bab847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.619447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075522ad-9ba3-4341-91bc-ae1b26bab847 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.630379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 992/1980] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:23.634987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.646376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09fd42-0e6f-43ea-a70f-124622a015ac admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.651620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.652951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.653242 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.653524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] PATH_INFO: `/v3/projects/4e57a75632644a3ab09e7fcbc996b363/users/86d8ecde67484529afe6c02d7b0e6a5f/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.655740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e57a75632644a3ab09e7fcbc996b363, user_id=86d8ecde67484529afe6c02d7b0e6a5f, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.658421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.674306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 989/1981] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:23 2022] PUT /identity/v3/projects/4e57a75632644a3ab09e7fcbc996b363/users/86d8ecde67484529afe6c02d7b0e6a5f/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:23.686036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.686332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.686618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.694852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:23.726683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] MFA Rules not processed for user `86d8ecde67484529afe6c02d7b0e6a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:23.728304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:23.827734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 993/1982] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:23.836835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:23.839776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:23.920605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd73f02-d0ad-465f-8b04-7f7b1cd98806 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:23.926809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add96354-a1eb-469a-81c7-59be29ed3de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:23.927522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add96354-a1eb-469a-81c7-59be29ed3de3 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:23.927802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add96354-a1eb-469a-81c7-59be29ed3de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:23.928075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add96354-a1eb-469a-81c7-59be29ed3de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:23.938529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add96354-a1eb-469a-81c7-59be29ed3de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:23.940013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add96354-a1eb-469a-81c7-59be29ed3de3 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:23.951981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 990/1983] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:24.031489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:24.145074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb22a8a-bf93-4f5c-a021-8ef68c5d5b73 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:24.153057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:24.153973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:24.154255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:24.154535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] PATH_INFO: `/v3/users/010a59e6fa2f4953aed9bf726d1815c6` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:24.155880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010a59e6fa2f4953aed9bf726d1815c6)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:24.158245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:24.233949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010a59e6fa2f4953aed9bf726d1815c6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.250427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010a59e6fa2f4953aed9bf726d1815c6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.279805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010a59e6fa2f4953aed9bf726d1815c6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.280794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010a59e6fa2f4953aed9bf726d1815c6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.281765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010a59e6fa2f4953aed9bf726d1815c6'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.312915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 994/1984] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:24 2022] DELETE /identity/v3/users/010a59e6fa2f4953aed9bf726d1815c6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:24.318665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:24.318957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:24.319343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:24.338223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:24.339514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:24.411089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 991/1985] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:24.421240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:24.422713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:24.433764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:24.485743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576235e-430e-4b89-883f-6df8364a0013 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:24.489855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:24.490519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:24.490806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:24.491105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:24.501907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:24.502762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:24.514565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 995/1986] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:24.779443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:24.781401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:24.794645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c5bf4-e8c5-4723-9fac-228991b8fb9d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:24.799821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:24.800875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:24.801247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:24.802140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] PATH_INFO: `/v3/users/c7563df960b54fbdbe083b60f91971a6` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:24.803621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7563df960b54fbdbe083b60f91971a6)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:24.805851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:24.833765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:24.856394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:24.858217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:24.879231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7563df960b54fbdbe083b60f91971a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.896032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7563df960b54fbdbe083b60f91971a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.903141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab72ee-8f6f-49cb-a283-b002017b5def service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:24.906136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7563df960b54fbdbe083b60f91971a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.913173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:24.914070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:24.914637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:24.915111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] PATH_INFO: `/v3/projects/0d1802ddd5094f44aeb6f56639aad876` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:24.923960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7563df960b54fbdbe083b60f91971a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.924127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d1802ddd5094f44aeb6f56639aad876)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:24.924812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ee4d5275-6ae0-4103-91db-dcc65c4ed355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7563df960b54fbdbe083b60f91971a6'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.925983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:24.932661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 992/1987] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:24 2022] DELETE /identity/v3/users/c7563df960b54fbdbe083b60f91971a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:24.937261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:24.937599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:24.937882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:24.958798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:24.960060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:24.990842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d1802ddd5094f44aeb6f56639aad876', 'request_id': 'req-8a40543c-ac43-4799-8c6c-734625502623'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:24.998039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d1802ddd5094f44aeb6f56639aad876', 'request_id': 'req-8a40543c-ac43-4799-8c6c-734625502623'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.004325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d1802ddd5094f44aeb6f56639aad876', 'request_id': 'req-8a40543c-ac43-4799-8c6c-734625502623'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.020839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] The token cache is being invalidate because project 0d1802ddd5094f44aeb6f56639aad876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:25.021133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.021793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.023667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 996/1988] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:24 2022] DELETE /identity/v3/projects/0d1802ddd5094f44aeb6f56639aad876 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:25.027460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:25.028846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:25.036639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 993/1989] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:25.048780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:25.050501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:25.120171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40543c-ac43-4799-8c6c-734625502623 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:25.125569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:25.126461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.126742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.127012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] PATH_INFO: `/v3/users/86d8ecde67484529afe6c02d7b0e6a5f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.128664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d8ecde67484529afe6c02d7b0e6a5f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:25.130921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:25.131886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4b5e6-bac2-4069-a4c5-4ef557eefed7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:25.174457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:25.187865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d8ecde67484529afe6c02d7b0e6a5f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.190743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:25.191436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.191724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.191989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.200408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:25.200956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d8ecde67484529afe6c02d7b0e6a5f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.202976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:25.204534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:25.209343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d8ecde67484529afe6c02d7b0e6a5f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.210249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d8ecde67484529afe6c02d7b0e6a5f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.210977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d8ecde67484529afe6c02d7b0e6a5f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.219564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 994/1990] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:25.229081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 997/1991] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:25 2022] DELETE /identity/v3/users/86d8ecde67484529afe6c02d7b0e6a5f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:25.630468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:25.632251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:25.643042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:25.644741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:25.651794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda5f93-d0e7-4ebb-b45c-0862f7e5c0bc service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:25.748629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:25.749500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.749784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.750055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] PATH_INFO: `/v3/projects/48b54332d600458e9a989ed07ee8e4b2` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.751287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b54332d600458e9a989ed07ee8e4b2)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:25.751446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1311bb-fe48-4c74-a116-c990ff8a3528 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:25.753257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:25.756338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:25.757187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.757464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.757732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] PATH_INFO: `/v3/projects/4e57a75632644a3ab09e7fcbc996b363` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.758978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e57a75632644a3ab09e7fcbc996b363)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:25.761462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:25.809739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b54332d600458e9a989ed07ee8e4b2', 'request_id': 'req-80eefec7-5791-4c63-919f-beaa57397ada'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.815881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e57a75632644a3ab09e7fcbc996b363', 'request_id': 'req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.816042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b54332d600458e9a989ed07ee8e4b2', 'request_id': 'req-80eefec7-5791-4c63-919f-beaa57397ada'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.822616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b54332d600458e9a989ed07ee8e4b2', 'request_id': 'req-80eefec7-5791-4c63-919f-beaa57397ada'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.822986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e57a75632644a3ab09e7fcbc996b363', 'request_id': 'req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.830000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e57a75632644a3ab09e7fcbc996b363', 'request_id': 'req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.837730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] The token cache is being invalidate because project 48b54332d600458e9a989ed07ee8e4b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:25.838033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.838665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-80eefec7-5791-4c63-919f-beaa57397ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.840282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 995/1992] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:25 2022] DELETE /identity/v3/projects/48b54332d600458e9a989ed07ee8e4b2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:25.845166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d26ee-32ef-4909-ad27-daeaaa4c8391 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.845459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d26ee-32ef-4909-ad27-daeaaa4c8391 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.845767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d26ee-32ef-4909-ad27-daeaaa4c8391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.846773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] The token cache is being invalidate because project 4e57a75632644a3ab09e7fcbc996b363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:25.847113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.847748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec46fac4-456c-43d8-9ed9-ec90f8091ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:25.850070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 998/1993] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:25 2022] DELETE /identity/v3/projects/4e57a75632644a3ab09e7fcbc996b363 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:25.855674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45435950-9a11-49c0-a2fd-22435fe44246 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.855976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45435950-9a11-49c0-a2fd-22435fe44246 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.856291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45435950-9a11-49c0-a2fd-22435fe44246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.866933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-923d26ee-32ef-4909-ad27-daeaaa4c8391 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:25.868216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-923d26ee-32ef-4909-ad27-daeaaa4c8391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:25.877042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-45435950-9a11-49c0-a2fd-22435fe44246 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:25.878258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-45435950-9a11-49c0-a2fd-22435fe44246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:25.923669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:25.945451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 996/1994] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:25.954819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.955202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.955492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.960773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 999/1995] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:25.969475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:25.969769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:25.970190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:25.982529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:25.983937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:26.000374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 997/1996] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:26.000654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:26.000654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:26.001132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.010492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1000/1997] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:26.015553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ba079-c1a3-435f-94ac-bbdecc5c7018 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.016255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.020947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.022277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.022564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.022840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.023647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:26.023955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.025566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.027612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4453eab4-392a-4725-af1d-7fde4d00279c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.033251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.034188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.034563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.035328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.036123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:26.036123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.040314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.047111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 998/1998] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:54:26.053785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.054292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1001/1999] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:26.058215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.070287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f595e-bfe6-4462-8f6e-dfbb3505295f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.072157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b74314-e40e-4fa5-877a-019d592d4cdd admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.075718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.075718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.075958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.079266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.079266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.079266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.084959 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.085916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.086303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.086697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.087602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.090810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.095961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:26.117335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1002/2000] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:26.120492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.126605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 999/2001] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:26.131516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.229062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac86e8-3be5-4b5f-bdf9-cd9911728853 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.248509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130d4ee-34ca-48aa-bc4f-11862b963dbb admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.255522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.256092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.256377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.256648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.257506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.260064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.290484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1000/2002] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:26.291704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.312744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffb10e-6f53-4a4c-9422-43a1d347ffb2 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.316272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.316851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.317696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.317977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.318683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.321016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.323455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.324496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.324936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.325719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.326709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.328782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.346895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1001/2003] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:26.354208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1003/2004] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:26.354208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.355622 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.372036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a2a1b-edc1-43a8-a0e9-3fbf019250c4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.372479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292c33d-2248-4661-88b4-37ba515a51e5 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.378022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.379077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.379442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.379615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.380551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.381215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.381339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.381594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.381701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.382327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.383553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.384034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.397600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1004/2005] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:54:26.403848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.404761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1002/2006] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:26.408098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.418198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91238ea0-264a-4474-8b3d-49459d4b8ac4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.424852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5827856-717d-4a7f-a581-1f42093cdca9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.425392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.426887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.427321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.427693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.428706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.430436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.432662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d0677-f78f-47b8-88ed-d28d22498fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.433584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d0677-f78f-47b8-88ed-d28d22498fa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.433868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d0677-f78f-47b8-88ed-d28d22498fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.434147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d0677-f78f-47b8-88ed-d28d22498fa4 admin admin] PATH_INFO: `/v3/projects/94f303a44f7b4916a06c55e64ed2dc49/users/cf13b31229794b81b9e6846c9bae8625/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.445317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1005/2007] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:26.451122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.458368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d0677-f78f-47b8-88ed-d28d22498fa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f303a44f7b4916a06c55e64ed2dc49, user_id=cf13b31229794b81b9e6846c9bae8625, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.461517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d0677-f78f-47b8-88ed-d28d22498fa4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.466163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa1a9a-4071-4a60-a17b-e02a59390790 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.473032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84111-1ea9-4a81-8aaf-9f0d25a78e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.474720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84111-1ea9-4a81-8aaf-9f0d25a78e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.474720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84111-1ea9-4a81-8aaf-9f0d25a78e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.475213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84111-1ea9-4a81-8aaf-9f0d25a78e9a admin admin] PATH_INFO: `/v3/projects/3847ae1ff5b540be933de619298b18b7/users/c1708eb483bc4cf78d68b5fe5e11a9ce/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.483584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1003/2008] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:26 2022] PUT /identity/v3/projects/94f303a44f7b4916a06c55e64ed2dc49/users/cf13b31229794b81b9e6846c9bae8625/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:26.494973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84111-1ea9-4a81-8aaf-9f0d25a78e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3847ae1ff5b540be933de619298b18b7, user_id=c1708eb483bc4cf78d68b5fe5e11a9ce, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.495112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.495207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.495475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.499297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84111-1ea9-4a81-8aaf-9f0d25a78e9a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.502235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:26.526749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1006/2009] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:26 2022] PUT /identity/v3/projects/3847ae1ff5b540be933de619298b18b7/users/c1708eb483bc4cf78d68b5fe5e11a9ce/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:26.538581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.538581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.538897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.550401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] MFA Rules not processed for user `cf13b31229794b81b9e6846c9bae8625`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:26.554371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:26.572126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] MFA Rules not processed for user `c1708eb483bc4cf78d68b5fe5e11a9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:26.573475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:26.662751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1004/2010] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:26.672318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.673571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1007/2011] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:26.674046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:26.679609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.733734 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:26.776095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea0768-4779-462c-a537-73bdcb69e11f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.781163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.781929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.782279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.782561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.784242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.787137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.816207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486acdfc-8a1c-40dc-b940-d1c858cb28a5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.818069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1008/2012] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:26.822254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.909572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f20e8-9544-4e5c-9317-e8e9709596ac admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.914145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.915265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.915477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.915933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.916067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.916258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.916532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.916801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.917240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.919284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.939217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:26.948707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1009/2013] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:26 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:26.952693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:26.969347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b2b8c-87e6-4be0-a75b-fd1bfe427bbd admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:26.978242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:26.979322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:26.980413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:26.980906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:26.981840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:26.984161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:26.999457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1010/2014] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:27.000536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:27.008484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:27.010553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:27.013357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c45d0-be23-4a32-aa61-9814731ae7e2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:27.018457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:27.019446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:27.019794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:27.020149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] PATH_INFO: `/v3/projects/97d9e566eadb42f0a70c7feddfa617df/users/b11cde97ac6b4386b22ec235b28f0c16/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:27.021041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1005/2015] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:54:27.040145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d9e566eadb42f0a70c7feddfa617df, user_id=b11cde97ac6b4386b22ec235b28f0c16, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:27.045300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:27.064992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1011/2016] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:26 2022] PUT /identity/v3/projects/97d9e566eadb42f0a70c7feddfa617df/users/b11cde97ac6b4386b22ec235b28f0c16/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:27.070249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:27.084218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6bdd1-38b2-474e-8d64-e789c713f23b service glance] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:27.178807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff68d-33f1-47c4-9677-2b0bea195a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:27.180629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff68d-33f1-47c4-9677-2b0bea195a19 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:27.181009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff68d-33f1-47c4-9677-2b0bea195a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:27.181404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff68d-33f1-47c4-9677-2b0bea195a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:27.182770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff68d-33f1-47c4-9677-2b0bea195a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:27.185693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ff68d-33f1-47c4-9677-2b0bea195a19 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:27.205356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1006/2017] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:27.206726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:27.222820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cda1c1-dc3d-4ce5-b8b7-d6bf49c7448c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:27.234350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:27.235606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:27.236091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:27.236659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] PATH_INFO: `/v3/projects/97d9e566eadb42f0a70c7feddfa617df/users/b11cde97ac6b4386b22ec235b28f0c16/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:27.240002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d9e566eadb42f0a70c7feddfa617df, user_id=b11cde97ac6b4386b22ec235b28f0c16, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:27.243807 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:27.263712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1012/2018] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:27 2022] PUT /identity/v3/projects/97d9e566eadb42f0a70c7feddfa617df/users/b11cde97ac6b4386b22ec235b28f0c16/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:27.274849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:27.275176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:27.275461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:27.288515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:27.320618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] MFA Rules not processed for user `b11cde97ac6b4386b22ec235b28f0c16`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:27.330896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:27.426031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1007/2019] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:27.439368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:27.441452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:27.518168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb94cb1-3855-4611-9667-c9e6922b7492 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:27.522976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:27.524028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:27.524470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:27.525100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:27.538732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:27.540489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:27.560543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1013/2020] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:27.983075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:27.993320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07b89b-0f41-491c-b330-00aa65eefd3b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:27.998842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b11cde97ac6b4386b22ec235b28f0c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97d9e566eadb42f0a70c7feddfa617df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:28.000119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:28.000119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:28.000386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:28.001240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:28.001658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:28.003583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:28.024756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1008/2021] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:28.026232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:28.036779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513f07e-4ba4-4e3a-a26b-54e44fcb68a5 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:28.041677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b11cde97ac6b4386b22ec235b28f0c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97d9e566eadb42f0a70c7feddfa617df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:28.042444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:28.042824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:28.043198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:28.043987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:28.045976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:28.075533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1014/2022] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:28 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:28.084726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:28.096043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f9a31-5ef2-4483-8181-24b78029e2dd tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:28.163404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b11cde97ac6b4386b22ec235b28f0c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97d9e566eadb42f0a70c7feddfa617df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:28.164100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:28.164388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:28.165474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:28.166241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:28.166563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:28.168245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:28.181401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1009/2023] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:28.220019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:28.249741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d373-3c37-4851-905b-c8959fc4bee9 tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:28.252423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b11cde97ac6b4386b22ec235b28f0c16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97d9e566eadb42f0a70c7feddfa617df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:28.253316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:28.253604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:28.253921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] PATH_INFO: `/v3/projects/a423c8d398cc4e92974716434ff3fc43` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:28.255664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a423c8d398cc4e92974716434ff3fc43)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:28.257649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:28.311756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a423c8d398cc4e92974716434ff3fc43', 'request_id': 'req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:28.321270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a423c8d398cc4e92974716434ff3fc43', 'request_id': 'req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:28.328380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a423c8d398cc4e92974716434ff3fc43', 'request_id': 'req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:28.346833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] The token cache is being invalidate because project a423c8d398cc4e92974716434ff3fc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:28.347352 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:28.348145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef47a0d8-5422-4f5d-8422-70d6c1efc04e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:28.349978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1015/2024] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:28 2022] DELETE /identity/v3/projects/a423c8d398cc4e92974716434ff3fc43 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:28.648441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:28.650519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:28.774087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a487b72-7b28-4f39-a07b-5d47b49bbb6e tempest-VolumeQuotaClassesTest-339876594 tempest-VolumeQuotaClassesTest-339876594-project-admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:28.796648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:28.883540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:28.883540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:28.884388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:28.884763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] PATH_INFO: `/v3/users/c1708eb483bc4cf78d68b5fe5e11a9ce` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:28.886336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1708eb483bc4cf78d68b5fe5e11a9ce)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:28.888660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:28.972840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1708eb483bc4cf78d68b5fe5e11a9ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:28.990779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1708eb483bc4cf78d68b5fe5e11a9ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.001409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1708eb483bc4cf78d68b5fe5e11a9ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.012513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1708eb483bc4cf78d68b5fe5e11a9ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.013989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1708eb483bc4cf78d68b5fe5e11a9ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.022636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1010/2025] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:28 2022] DELETE /identity/v3/users/c1708eb483bc4cf78d68b5fe5e11a9ce => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:29.027441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:29.028030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:29.028132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:29.069527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:29.071780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:29.085804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:29.151501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1016/2026] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:29.161682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:29.163322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:29.226910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42bc1b-7766-44cf-8523-78d55d1ad57c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:29.230670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:29.231368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:29.231658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:29.231943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:29.243621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:29.245559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:29.266693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1011/2027] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:29.612439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:29.615969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:29.633941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525cc41-252e-4e65-9b91-36e097da6b2f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:29.645153 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:29.646164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:29.646519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:29.646985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] PATH_INFO: `/v3/projects/3847ae1ff5b540be933de619298b18b7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:29.648615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3847ae1ff5b540be933de619298b18b7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:29.650509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:29.707391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3847ae1ff5b540be933de619298b18b7', 'request_id': 'req-9a85b65e-644e-4467-9829-3482fe2f7d7c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.714655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3847ae1ff5b540be933de619298b18b7', 'request_id': 'req-9a85b65e-644e-4467-9829-3482fe2f7d7c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.733580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3847ae1ff5b540be933de619298b18b7', 'request_id': 'req-9a85b65e-644e-4467-9829-3482fe2f7d7c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.752230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] The token cache is being invalidate because project 3847ae1ff5b540be933de619298b18b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:29.752230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.753004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.754916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1017/2028] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:29 2022] DELETE /identity/v3/projects/3847ae1ff5b540be933de619298b18b7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:29.759267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:29.770311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebbae6-7479-4a9c-b737-fddaae05cebc service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:29.821782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:29.863643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:29.866378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:29.866974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:29.867460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] PATH_INFO: `/v3/users/b11cde97ac6b4386b22ec235b28f0c16` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:29.869635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11cde97ac6b4386b22ec235b28f0c16)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:29.875380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:29.953233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11cde97ac6b4386b22ec235b28f0c16'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.967919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11cde97ac6b4386b22ec235b28f0c16'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.982814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11cde97ac6b4386b22ec235b28f0c16'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.992386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11cde97ac6b4386b22ec235b28f0c16'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:29.993128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-eef90de4-1328-40f6-850f-eb3cf3340f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11cde97ac6b4386b22ec235b28f0c16'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:30.000315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1012/2029] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:29 2022] DELETE /identity/v3/users/b11cde97ac6b4386b22ec235b28f0c16 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:30.345175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:30.346880 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:30.374040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a85b65e-644e-4467-9829-3482fe2f7d7c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:30.462294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:30.463416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:30.463694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:30.463968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] PATH_INFO: `/v3/projects/97d9e566eadb42f0a70c7feddfa617df` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:30.465332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d9e566eadb42f0a70c7feddfa617df)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:30.467580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:30.517025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:30.526377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d9e566eadb42f0a70c7feddfa617df', 'request_id': 'req-21f6dd7f-5fa2-421d-9b2d-059d404564dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:30.533486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d9e566eadb42f0a70c7feddfa617df', 'request_id': 'req-21f6dd7f-5fa2-421d-9b2d-059d404564dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:30.540097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d9e566eadb42f0a70c7feddfa617df', 'request_id': 'req-21f6dd7f-5fa2-421d-9b2d-059d404564dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:30.556601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] The token cache is being invalidate because project 97d9e566eadb42f0a70c7feddfa617df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:30.556885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:30.557539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-21f6dd7f-5fa2-421d-9b2d-059d404564dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:30.559439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1018/2030] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:30 2022] DELETE /identity/v3/projects/97d9e566eadb42f0a70c7feddfa617df => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:30.564173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:30.564551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:30.564853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:30.589371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:30.590748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:30.661357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1013/2031] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:30.670257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:30.670543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:30.670824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:30.691244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:30.693137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:30.701271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1019/2032] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:30.705537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:30.720583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54cbfda-acfe-4d30-805b-ba4f55f8f33b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:30.728028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:30.728718 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:30.729005 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:30.729282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:30.730038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:30.730352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:30.732034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:30.754255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1014/2033] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:30.758704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:30.768896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22f512-009c-4810-aae0-fc1f02865408 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:30.778370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:30.780125 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:30.782793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:30.783209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:30.783975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:30.786991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:30.831618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1020/2034] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:30 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:30.836190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:30.861452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9ba81-056a-4dbb-97f8-76fb3920b9c2 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:30.945918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:30.968995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:30.970253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:30.970878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:30.971751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:30.972474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:30.975560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.002094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1015/2035] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:30 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:31.006371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.019258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c430f-13e1-4fdd-93a0-24cf9dade660 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.024905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.025665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.026043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.026382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.027265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.028956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.041085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1021/2036] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:31.045876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.056652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f18f6-ba57-47f0-9cb7-2f8eed04a3d5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.061620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86482023-8a23-4354-8475-59ef9ee94444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.062304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86482023-8a23-4354-8475-59ef9ee94444 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.062591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86482023-8a23-4354-8475-59ef9ee94444 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.062867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86482023-8a23-4354-8475-59ef9ee94444 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.063648 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86482023-8a23-4354-8475-59ef9ee94444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.065267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86482023-8a23-4354-8475-59ef9ee94444 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.080466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1016/2037] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:54:31.083498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.100329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944023e-aa6d-4b9d-8d2e-9f38daa7e7d8 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.105381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.106276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.106559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.106839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] PATH_INFO: `/v3/projects/d0f7332548bb4070bd1bac2907ae9bb4/users/4942fa9f821941ca9ec7f31c04ef13c4/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.125947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f7332548bb4070bd1bac2907ae9bb4, user_id=4942fa9f821941ca9ec7f31c04ef13c4, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.129051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.156946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1022/2038] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:31 2022] PUT /identity/v3/projects/d0f7332548bb4070bd1bac2907ae9bb4/users/4942fa9f821941ca9ec7f31c04ef13c4/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:31.170576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.171174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.171671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.250936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] MFA Rules not processed for user `4942fa9f821941ca9ec7f31c04ef13c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:31.250936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:31.390322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1017/2039] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:31.399631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.502217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3ce3c-1a5a-4efc-9ded-c0af005f8776 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.508121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.508806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.509090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.509368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.510081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.512611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.549330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1023/2040] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:31 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:31.552552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.597399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea259d3-862b-4343-a50a-1d4991e3da22 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.725391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.726083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.726361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.726634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.727376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.729330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.769256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1018/2041] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:31 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:31.773180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.787766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09be9cb-1073-4558-b3ce-fce7d1eafe8c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.799692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.800409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.800907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.801785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.802801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.808147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.822939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1024/2042] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:54:31.825239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.840250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c9ce7-3bed-4af3-911c-c8742628f954 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.861837 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:31.863115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:31.864768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:31.866787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] PATH_INFO: `/v3/projects/c12595946c6f4f95b92a640ff705a58f/users/1fdf2d539faa48fbb8584acc16946f58/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:31.904404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12595946c6f4f95b92a640ff705a58f, user_id=1fdf2d539faa48fbb8584acc16946f58, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:31.907739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:31.925819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1019/2043] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:31 2022] PUT /identity/v3/projects/c12595946c6f4f95b92a640ff705a58f/users/1fdf2d539faa48fbb8584acc16946f58/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:31.930451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:31.941707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb8f56-08e2-4e77-9743-58b39d1686af admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:31.994894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:32.024765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:32.025944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.026545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.027221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.027708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.028199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.029151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.031160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.043428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1025/2044] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:32.048059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:32.054914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d3272-27b5-4341-ba8c-8ce2816c74e9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:32.061194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ebbd6-8ffd-4bde-aecf-4c5e57c37083 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:32.065505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.067098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.067589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.068038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] PATH_INFO: `/v3/users/74e755a5fe9f4c5c95376beae691381f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.069402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74e755a5fe9f4c5c95376beae691381f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.071682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.079198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cc375-ad16-4f23-8d5b-b033cd1fefc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.079198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cc375-ad16-4f23-8d5b-b033cd1fefc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.079198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cc375-ad16-4f23-8d5b-b033cd1fefc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.079198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cc375-ad16-4f23-8d5b-b033cd1fefc8 admin admin] PATH_INFO: `/v3/projects/c12595946c6f4f95b92a640ff705a58f/users/1fdf2d539faa48fbb8584acc16946f58/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.083725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cc375-ad16-4f23-8d5b-b033cd1fefc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12595946c6f4f95b92a640ff705a58f, user_id=1fdf2d539faa48fbb8584acc16946f58, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.086750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cc375-ad16-4f23-8d5b-b033cd1fefc8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.096628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:INSERT=7,keystone:DELETE=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:32.099841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1026/2045] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:32 2022] PUT /identity/v3/projects/c12595946c6f4f95b92a640ff705a58f/users/1fdf2d539faa48fbb8584acc16946f58/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:32.112872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.113355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.114403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.149235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] MFA Rules not processed for user `1fdf2d539faa48fbb8584acc16946f58`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:32.150480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.154947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74e755a5fe9f4c5c95376beae691381f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.173816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74e755a5fe9f4c5c95376beae691381f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.183635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74e755a5fe9f4c5c95376beae691381f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.205027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74e755a5fe9f4c5c95376beae691381f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.206072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74e755a5fe9f4c5c95376beae691381f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.215224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1020/2046] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:32 2022] DELETE /identity/v3/users/74e755a5fe9f4c5c95376beae691381f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:32.240595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1027/2047] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:32.252082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:32.253657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.321657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126ac13-7314-4f49-8e13-0284e2bd3827 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:32.325369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.326197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.326511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.326780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.328668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.399386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.400863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf09c9-b179-4911-b286-03cb50ed449f service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.407030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:32.412748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.414678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1021/2048] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:54:32.518437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2cf3f-ad2f-4017-9e32-20394da3ee62 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:32.603012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:32.619798 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.620865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.621177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.621474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] PATH_INFO: `/v3/projects/932a50a36025440583c478450e42f9e8` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.622707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=932a50a36025440583c478450e42f9e8)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.624656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.693684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932a50a36025440583c478450e42f9e8', 'request_id': 'req-34d2781c-2074-4e6f-bef0-12341fab7d4e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.701537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932a50a36025440583c478450e42f9e8', 'request_id': 'req-34d2781c-2074-4e6f-bef0-12341fab7d4e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.711122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932a50a36025440583c478450e42f9e8', 'request_id': 'req-34d2781c-2074-4e6f-bef0-12341fab7d4e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.730787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] The token cache is being invalidate because project 932a50a36025440583c478450e42f9e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:32.731301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.732735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-34d2781c-2074-4e6f-bef0-12341fab7d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:32.734834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1028/2049] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:32 2022] DELETE /identity/v3/projects/932a50a36025440583c478450e42f9e8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:32.740889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.741179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.741725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.773601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:32.775516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.838954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:32.861920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1022/2050] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:32.871343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.871627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.871904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.890647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:32.891971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:32.900130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1029/2051] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:32.904415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:32.916261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5d04c-d7d5-41de-beec-6bb75bf4e340 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:32.921356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.922066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.922350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.922629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.923418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:32.923747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.925356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.939843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1023/2052] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:32.944268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:32.954096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0c692-0e9f-4ad4-a5fd-85b580105b9a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:32.960260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:32.961569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:32.962134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:32.962817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:32.963981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:32.966026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:32.995943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1030/2053] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:32 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:33.001582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.012639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8245f-053d-4ef7-a986-6925f3baf4d3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.078420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.079151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.079413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.079689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.080408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.082375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.110544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1024/2054] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:33 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:33.115846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.127403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad1ea8-4aaa-4e85-9de3-e5ae036479d8 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.132838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.133517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.133793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.134066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.134810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.136842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.149111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1031/2055] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:33.153237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.164495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5593f0-859d-4429-a8ac-cff7f1e1a261 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.169303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a385b0-1588-407c-acd6-b1aca0ac7f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.169980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a385b0-1588-407c-acd6-b1aca0ac7f1f admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.170293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a385b0-1588-407c-acd6-b1aca0ac7f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.170571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a385b0-1588-407c-acd6-b1aca0ac7f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.171345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a385b0-1588-407c-acd6-b1aca0ac7f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.172960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a385b0-1588-407c-acd6-b1aca0ac7f1f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.184031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1025/2056] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:33.188328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.198891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248993e7-b0ba-46b1-843a-901167cf5222 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.204123 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.205057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.205335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.205608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] PATH_INFO: `/v3/projects/d0252d8dba314ba6a9e7f137c150dfc0/users/94779175caf449f3beca2ae141815f89/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.221379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0252d8dba314ba6a9e7f137c150dfc0, user_id=94779175caf449f3beca2ae141815f89, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.224699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.240617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1032/2057] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:33 2022] PUT /identity/v3/projects/d0252d8dba314ba6a9e7f137c150dfc0/users/94779175caf449f3beca2ae141815f89/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:33.250979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.251311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.251625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.280905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] MFA Rules not processed for user `94779175caf449f3beca2ae141815f89`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:33.282129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:33.369708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1026/2058] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:33.370349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.447913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466491bb-f782-411c-bce0-7a9030f7b057 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.455604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.456284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.456565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.456850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.457653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.460227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.491793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1033/2059] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:33 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:33.496771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.508980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4851e-d6e5-4fe0-9bcd-1e59831ad468 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.552614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:33.573079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.573754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.574034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.574317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.575022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.576972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.598487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1027/2060] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:33 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:33.602553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.609428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:33.613468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a23cc5-bb68-4de1-bfef-cf01f37cf5be admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.620662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.621281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.621564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.621900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.622622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.624882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.637157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1034/2061] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:33.641581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.652910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e1be0-ca9e-4e72-b386-783ae84a8bcc admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.657607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.658515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.658820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.659095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] PATH_INFO: `/v3/projects/231eb8b9a74f412c9487fb340b4891f8/users/4f3508f8f64d472ea8c421a1dcd0d937/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.675922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231eb8b9a74f412c9487fb340b4891f8, user_id=4f3508f8f64d472ea8c421a1dcd0d937, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.678674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.695344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1028/2062] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:33 2022] PUT /identity/v3/projects/231eb8b9a74f412c9487fb340b4891f8/users/4f3508f8f64d472ea8c421a1dcd0d937/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:33.700739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.713349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000011a2-439d-4a84-b45c-0fdad3f8abb9 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.777298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf42ac5-da79-42ac-a636-fb730ff18814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.777973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42ac5-da79-42ac-a636-fb730ff18814 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.778254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42ac5-da79-42ac-a636-fb730ff18814 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.778526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42ac5-da79-42ac-a636-fb730ff18814 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.779315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf42ac5-da79-42ac-a636-fb730ff18814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.780908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf42ac5-da79-42ac-a636-fb730ff18814 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.800951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1035/2063] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:33.803043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:33.820140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b8f50-91f5-4f23-b10f-06de091952b0 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:33.833050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:33.833336 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.833820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.834098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] PATH_INFO: `/v3/projects/231eb8b9a74f412c9487fb340b4891f8/users/4f3508f8f64d472ea8c421a1dcd0d937/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.836627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231eb8b9a74f412c9487fb340b4891f8, user_id=4f3508f8f64d472ea8c421a1dcd0d937, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:33.839909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:33.854687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1029/2064] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:33 2022] PUT /identity/v3/projects/231eb8b9a74f412c9487fb340b4891f8/users/4f3508f8f64d472ea8c421a1dcd0d937/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:33.868127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:33.868647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:33.869386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:33.916023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] MFA Rules not processed for user `4f3508f8f64d472ea8c421a1dcd0d937`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:33.917295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:34.027119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1036/2065] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:34.040824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:34.042567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:34.108261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17983e0-5930-4232-ac85-a37ee24a3ac5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:34.113729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:34.113729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:34.113729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:34.113729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:34.124470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:34.125924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:34.140349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1030/2066] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:34.243254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:34.253374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf8c9e-4705-4c12-87fd-50055dd7c602 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:34.257795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ebf635-3fc7-4d34-8f45-9e8aa62f01c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:34.258446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf635-3fc7-4d34-8f45-9e8aa62f01c0 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:34.258718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf635-3fc7-4d34-8f45-9e8aa62f01c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:34.258995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ebf635-3fc7-4d34-8f45-9e8aa62f01c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:34.324859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebf635-3fc7-4d34-8f45-9e8aa62f01c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:34.326337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ebf635-3fc7-4d34-8f45-9e8aa62f01c0 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:34.338145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1037/2067] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:35.193570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:35.195046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:35.272507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afaad9-159e-422a-afff-42b52e894f5c service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:35.277362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:35.278333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:35.278563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:35.278841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] PATH_INFO: `/v3/users/4942fa9f821941ca9ec7f31c04ef13c4` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:35.280115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4942fa9f821941ca9ec7f31c04ef13c4)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:35.282070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:35.325666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:35.335937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4942fa9f821941ca9ec7f31c04ef13c4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:35.348770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4942fa9f821941ca9ec7f31c04ef13c4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:35.355800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4942fa9f821941ca9ec7f31c04ef13c4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:35.364088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4942fa9f821941ca9ec7f31c04ef13c4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:35.364850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4942fa9f821941ca9ec7f31c04ef13c4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:35.371549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1031/2068] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:35 2022] DELETE /identity/v3/users/4942fa9f821941ca9ec7f31c04ef13c4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:35.376106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:35.376658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:35.376658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:35.393948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:35.395712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:35.404549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:35.459392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1038/2069] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:35.469284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:35.470779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:35.538980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74585d56-38af-467a-a882-f4741fb66c0d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:35.542876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:35.543998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:35.544584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:35.544994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:35.557025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:35.558492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:35.569825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1032/2070] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:35.905878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:35.907155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:35.932671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622039b-137f-4857-ac50-979489de2aa5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:35.945399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:35.946504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:35.947350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:35.947639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] PATH_INFO: `/v3/projects/d0f7332548bb4070bd1bac2907ae9bb4` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:35.949045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f7332548bb4070bd1bac2907ae9bb4)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:35.951090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:36.014271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f7332548bb4070bd1bac2907ae9bb4', 'request_id': 'req-051f901a-e71c-45d2-b03c-6786f21b64eb'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.024273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f7332548bb4070bd1bac2907ae9bb4', 'request_id': 'req-051f901a-e71c-45d2-b03c-6786f21b64eb'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.030385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f7332548bb4070bd1bac2907ae9bb4', 'request_id': 'req-051f901a-e71c-45d2-b03c-6786f21b64eb'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.043607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] The token cache is being invalidate because project d0f7332548bb4070bd1bac2907ae9bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:36.044701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.045365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.047073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1039/2071] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:35 2022] DELETE /identity/v3/projects/d0f7332548bb4070bd1bac2907ae9bb4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:36.050976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:36.062629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d2b9f-f7bb-44ad-bcab-838226af1109 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:36.128343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:36.129223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:36.130065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:36.130347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] PATH_INFO: `/v3/users/1fdf2d539faa48fbb8584acc16946f58` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:36.131644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fdf2d539faa48fbb8584acc16946f58)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:36.133571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:36.181315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fdf2d539faa48fbb8584acc16946f58'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.193340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:36.198800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fdf2d539faa48fbb8584acc16946f58'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.209258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fdf2d539faa48fbb8584acc16946f58'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.217881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fdf2d539faa48fbb8584acc16946f58'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.218539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ff2971a4-d967-4bb2-a578-3d7dfb26510a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fdf2d539faa48fbb8584acc16946f58'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.225918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1033/2072] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:36 2022] DELETE /identity/v3/users/1fdf2d539faa48fbb8584acc16946f58 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:36.619520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:36.621563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:36.635547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f901a-e71c-45d2-b03c-6786f21b64eb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:36.710944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:36.722563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:36.722563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:36.722563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] PATH_INFO: `/v3/projects/c12595946c6f4f95b92a640ff705a58f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:36.722563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12595946c6f4f95b92a640ff705a58f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:36.722563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:36.764590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:36.774513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12595946c6f4f95b92a640ff705a58f', 'request_id': 'req-7ba7b78a-fff8-47f7-ad03-d2bde7876038'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.781678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12595946c6f4f95b92a640ff705a58f', 'request_id': 'req-7ba7b78a-fff8-47f7-ad03-d2bde7876038'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.787862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12595946c6f4f95b92a640ff705a58f', 'request_id': 'req-7ba7b78a-fff8-47f7-ad03-d2bde7876038'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.802410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] The token cache is being invalidate because project c12595946c6f4f95b92a640ff705a58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:36.802855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.803554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:36.805174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1040/2073] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:36 2022] DELETE /identity/v3/projects/c12595946c6f4f95b92a640ff705a58f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:43.589680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9b09-4e20-4d00-a1b1-c5e28ad9b2a7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:43.590223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9b09-4e20-4d00-a1b1-c5e28ad9b2a7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:43.590364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9b09-4e20-4d00-a1b1-c5e28ad9b2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:43.609238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-41fa9b09-4e20-4d00-a1b1-c5e28ad9b2a7 None None] MFA Rules not processed for user `9c901aba30d141b39e7189eefc7c1ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:43.610378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-41fa9b09-4e20-4d00-a1b1-c5e28ad9b2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:43.669876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1034/2074] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:45.560374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:45.561918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:45.642999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7b78a-fff8-47f7-ad03-d2bde7876038 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:45.649498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:45.650516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:45.651008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:45.651361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] PATH_INFO: `/v3/users/94779175caf449f3beca2ae141815f89` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:45.652736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94779175caf449f3beca2ae141815f89)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:45.654754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:45.721674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94779175caf449f3beca2ae141815f89'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:45.739653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94779175caf449f3beca2ae141815f89'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:45.757187 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94779175caf449f3beca2ae141815f89'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:45.757979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94779175caf449f3beca2ae141815f89'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:45.758771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94779175caf449f3beca2ae141815f89'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:45.777430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1041/2075] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:45 2022] DELETE /identity/v3/users/94779175caf449f3beca2ae141815f89 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:45.781806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd338358-261b-4819-8628-a18e804d4427 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:45.782324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd338358-261b-4819-8628-a18e804d4427 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:45.782435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd338358-261b-4819-8628-a18e804d4427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:45.810626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-bd338358-261b-4819-8628-a18e804d4427 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:45.812071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bd338358-261b-4819-8628-a18e804d4427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:45.896957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1035/2076] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:45.907561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:45.909169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:45.969884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:45.976429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986fefd-0126-4926-a36a-b28a1f9d3523 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:45.984003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:45.984691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:45.985217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:45.985433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:46.001690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:46.003215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:46.014754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1042/2077] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:46.341296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd338358-261b-4819-8628-a18e804d4427 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:46.343125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bd338358-261b-4819-8628-a18e804d4427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:46.359769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd338358-261b-4819-8628-a18e804d4427 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:46.366372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:46.367371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:46.368098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:46.368378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] PATH_INFO: `/v3/projects/d0252d8dba314ba6a9e7f137c150dfc0` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:46.370698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0252d8dba314ba6a9e7f137c150dfc0)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:46.374743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:46.442874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0252d8dba314ba6a9e7f137c150dfc0', 'request_id': 'req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.451704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0252d8dba314ba6a9e7f137c150dfc0', 'request_id': 'req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.459544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0252d8dba314ba6a9e7f137c150dfc0', 'request_id': 'req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.485088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] The token cache is being invalidate because project d0252d8dba314ba6a9e7f137c150dfc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:46.485524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.486756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.490461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1036/2078] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:46 2022] DELETE /identity/v3/projects/d0252d8dba314ba6a9e7f137c150dfc0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:46.494977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:46.506206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525675c-a9bd-4e26-9652-4206f1c5f6af service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:46.569256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:46.570145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:46.570422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:46.570691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] PATH_INFO: `/v3/users/4f3508f8f64d472ea8c421a1dcd0d937` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:46.571987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3508f8f64d472ea8c421a1dcd0d937)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:46.573953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:46.633821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3508f8f64d472ea8c421a1dcd0d937'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.659141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3508f8f64d472ea8c421a1dcd0d937'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.668340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3508f8f64d472ea8c421a1dcd0d937'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.669101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3508f8f64d472ea8c421a1dcd0d937'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.669724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-25ee9735-6f27-4852-abb8-6ae0b02ee5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3508f8f64d472ea8c421a1dcd0d937'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:46.686660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1043/2079] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:54:46 2022] DELETE /identity/v3/users/4f3508f8f64d472ea8c421a1dcd0d937 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:47.006937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.013897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:47.027973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde1b1a-5abd-46d0-9249-4ea33d2a9f73 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.033828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:47.126905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.128060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.128418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.128748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] PATH_INFO: `/v3/projects/231eb8b9a74f412c9487fb340b4891f8` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.130834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231eb8b9a74f412c9487fb340b4891f8)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.133195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.188267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231eb8b9a74f412c9487fb340b4891f8', 'request_id': 'req-5fde72cc-27da-405f-b2d3-6f5631186fe3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:47.199500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231eb8b9a74f412c9487fb340b4891f8', 'request_id': 'req-5fde72cc-27da-405f-b2d3-6f5631186fe3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:47.205460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231eb8b9a74f412c9487fb340b4891f8', 'request_id': 'req-5fde72cc-27da-405f-b2d3-6f5631186fe3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:47.221330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] The token cache is being invalidate because project 231eb8b9a74f412c9487fb340b4891f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:54:47.221621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:47.222297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5fde72cc-27da-405f-b2d3-6f5631186fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:54:47.224990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1037/2080] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:54:47 2022] DELETE /identity/v3/projects/231eb8b9a74f412c9487fb340b4891f8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:47.229882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.230264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.230613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.254024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:47.256458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:47.337288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1044/2081] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:54:47.345148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.345448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.345728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.386756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:47.389831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:47.401455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1038/2082] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:47.406332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.417391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7035-2440-4c40-b602-44ab8e931548 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.422723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.423447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.423725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.423999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.424749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:54:47.425056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.427870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.430583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:47.446868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1045/2083] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:54:47.447904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.458437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d183e-16bb-48c7-bd0d-e0508a615608 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.463234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.463909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.464199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.464474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.465176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.467110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.503565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1039/2084] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:47 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:54:47.505934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.516786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d096a6-c374-4bb1-8baf-7fbe5e6d1c97 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.582393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.583325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.583548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.583809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.584567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.586530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.611289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1046/2085] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:47 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:47.616438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.629089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880062-3510-40b6-b0a1-109ef701ec8a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.633979 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.634665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.634944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.635249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.635997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.637611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.650628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1040/2086] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:47.655098 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.666365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91753706-c944-467f-bd19-7399073c24f9 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.673337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c398a2-525e-4b83-8c2a-442aab876dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.674157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c398a2-525e-4b83-8c2a-442aab876dba admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.674514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c398a2-525e-4b83-8c2a-442aab876dba admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.674857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c398a2-525e-4b83-8c2a-442aab876dba admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.675785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c398a2-525e-4b83-8c2a-442aab876dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.678143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c398a2-525e-4b83-8c2a-442aab876dba admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.689084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1047/2087] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:47.693728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.705896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae41a98-aa33-4732-b978-f88a69bb9108 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.721342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.722316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.722741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.723922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] PATH_INFO: `/v3/projects/1c232fdec0284b368d3efe370378eb6f/users/291647dcb1824301a4e63fe56bd6e11a/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.742356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c232fdec0284b368d3efe370378eb6f, user_id=291647dcb1824301a4e63fe56bd6e11a, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.746197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:47.762690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1041/2088] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:47 2022] PUT /identity/v3/projects/1c232fdec0284b368d3efe370378eb6f/users/291647dcb1824301a4e63fe56bd6e11a/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:47.773846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.774137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.774420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.805594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] MFA Rules not processed for user `291647dcb1824301a4e63fe56bd6e11a`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:47.807541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:47.885430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1048/2089] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:47.889779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:47.945813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:47.982693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cbaf6-9392-4f8d-b735-17d28923fb41 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:47.987390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:47.988051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:47.988331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:47.989521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:47.989936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:47.991858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.019093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1042/2090] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:54:47 2022] POST /identity/v3/projects => generated 363 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:48.035000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.047076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971bf42-abec-4a61-9800-31eea1ed4d1b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.109354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.110033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.110317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.110597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.111359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.113430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.122867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:48.137245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1049/2091] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:54:48 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:54:48.141395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.154182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8371d73-5c4f-4b95-8f4a-b2ae057f27ab admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.159987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.161370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.161656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.162077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.162952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.164712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.176591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1043/2092] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:48.180720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.190943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e178eaa-f433-4361-9097-1d582e01b481 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.195787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.196712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.196999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.197276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] PATH_INFO: `/v3/projects/77bb1de8d8084f22b055d183420453d6/users/eea255908b1949568ebc629da0102154/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.215576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bb1de8d8084f22b055d183420453d6, user_id=eea255908b1949568ebc629da0102154, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.218303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.232143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1050/2093] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:48 2022] PUT /identity/v3/projects/77bb1de8d8084f22b055d183420453d6/users/eea255908b1949568ebc629da0102154/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:54:48.236260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.246617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa68c4f-6ee0-4beb-ac76-15c3fdbd51fd admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.306896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc806eb-43cd-4496-a0a2-e784304c6a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.307911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc806eb-43cd-4496-a0a2-e784304c6a45 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.308331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc806eb-43cd-4496-a0a2-e784304c6a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.308692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc806eb-43cd-4496-a0a2-e784304c6a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.309439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc806eb-43cd-4496-a0a2-e784304c6a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.311208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc806eb-43cd-4496-a0a2-e784304c6a45 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.323908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1044/2094] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:54:48.328190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.340210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39163a9d-4103-40cc-b19e-0e60233152fb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.344932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.345850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.346128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.346431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] PATH_INFO: `/v3/projects/77bb1de8d8084f22b055d183420453d6/users/eea255908b1949568ebc629da0102154/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.349379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bb1de8d8084f22b055d183420453d6, user_id=eea255908b1949568ebc629da0102154, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.352080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.363198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1051/2095] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:54:48 2022] PUT /identity/v3/projects/77bb1de8d8084f22b055d183420453d6/users/eea255908b1949568ebc629da0102154/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:54:48.381683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.381987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.382277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.411235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] MFA Rules not processed for user `eea255908b1949568ebc629da0102154`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:54:48.414048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:48.495099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1045/2096] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:54:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:54:48.506983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.508913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:54:48.570624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983278a7-b6cb-4d59-83fd-91f61d544deb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.574387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.575030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.575338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.575613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.585658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.587211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.597960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1052/2097] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:48.672086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:54:48.683834 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf9e3b-d073-4c8f-8b72-8440b0c1f95d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:54:48.690244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9636b5d-fa16-48d3-a887-b1565ddddd6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:54:48.691436 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9636b5d-fa16-48d3-a887-b1565ddddd6c service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:54:48.692458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9636b5d-fa16-48d3-a887-b1565ddddd6c service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:54:48.692960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9636b5d-fa16-48d3-a887-b1565ddddd6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:54:48.726002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:48.765992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9636b5d-fa16-48d3-a887-b1565ddddd6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:54:48.767421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9636b5d-fa16-48d3-a887-b1565ddddd6c service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:54:48.778275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1046/2098] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:54:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:54:58.592846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=3,keystone:SELECT=43 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:54:58.772582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=15 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:00.343105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:00.409667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34cbe33-0947-44fb-b0b6-623b19177c6a service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:00.415769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:00.416655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:00.416934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:00.417206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] PATH_INFO: `/v3/users/291647dcb1824301a4e63fe56bd6e11a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:00.418456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291647dcb1824301a4e63fe56bd6e11a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:00.420509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:00.471122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291647dcb1824301a4e63fe56bd6e11a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:00.483145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291647dcb1824301a4e63fe56bd6e11a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:00.489911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291647dcb1824301a4e63fe56bd6e11a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:00.490635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291647dcb1824301a4e63fe56bd6e11a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:00.491267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291647dcb1824301a4e63fe56bd6e11a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:00.507994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1053/2099] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:00 2022] DELETE /identity/v3/users/291647dcb1824301a4e63fe56bd6e11a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:00.512673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:00.513286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:00.513286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:00.531586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:00.532801 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:00.601372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1047/2100] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:00.612432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:00.614672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:00.702037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f192e8-3c7c-41bb-bfb4-352fea1c3e06 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:00.711829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:00.712536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:00.712950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:00.713244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:00.724279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:00.725893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:00.736833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1054/2101] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:01.111708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:01.111708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:01.124910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69240-9b1f-4396-8d95-8a453bef6820 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:01.131275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:01.132185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:01.132621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:01.132899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] PATH_INFO: `/v3/projects/1c232fdec0284b368d3efe370378eb6f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:01.134230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c232fdec0284b368d3efe370378eb6f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:01.136280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:01.191289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c232fdec0284b368d3efe370378eb6f', 'request_id': 'req-d8090748-d71a-49ff-a690-82ab1bd119ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.197567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c232fdec0284b368d3efe370378eb6f', 'request_id': 'req-d8090748-d71a-49ff-a690-82ab1bd119ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.203100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c232fdec0284b368d3efe370378eb6f', 'request_id': 'req-d8090748-d71a-49ff-a690-82ab1bd119ce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.215586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] The token cache is being invalidate because project 1c232fdec0284b368d3efe370378eb6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:01.215868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.216494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.218078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1048/2102] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:01 2022] DELETE /identity/v3/projects/1c232fdec0284b368d3efe370378eb6f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:01.223362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:01.233766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec225ce-5900-402a-bd61-6b4b755f83e5 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:01.253092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:01.294736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:01.295648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:01.295934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:01.296299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] PATH_INFO: `/v3/users/eea255908b1949568ebc629da0102154` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:01.297493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea255908b1949568ebc629da0102154)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:01.299459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:01.354888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea255908b1949568ebc629da0102154'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.368299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea255908b1949568ebc629da0102154'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.375615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea255908b1949568ebc629da0102154'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.376359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea255908b1949568ebc629da0102154'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.376996 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f5ad2235-35f7-4343-82d0-2b3504ee4180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea255908b1949568ebc629da0102154'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.390908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1055/2103] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:01 2022] DELETE /identity/v3/users/eea255908b1949568ebc629da0102154 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:01.663553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:01.665014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:01.680205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8090748-d71a-49ff-a690-82ab1bd119ce admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:01.743328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:01.744199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:01.744737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:01.745022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] PATH_INFO: `/v3/projects/77bb1de8d8084f22b055d183420453d6` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:01.746221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bb1de8d8084f22b055d183420453d6)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:01.748197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:01.779144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:01.797811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bb1de8d8084f22b055d183420453d6', 'request_id': 'req-08e01616-74d4-4872-8b7b-218d9533aa77'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.804597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bb1de8d8084f22b055d183420453d6', 'request_id': 'req-08e01616-74d4-4872-8b7b-218d9533aa77'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.810081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bb1de8d8084f22b055d183420453d6', 'request_id': 'req-08e01616-74d4-4872-8b7b-218d9533aa77'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.822547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] The token cache is being invalidate because project 77bb1de8d8084f22b055d183420453d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:01.822826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.823540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e01616-74d4-4872-8b7b-218d9533aa77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:01.825223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1049/2104] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:01 2022] DELETE /identity/v3/projects/77bb1de8d8084f22b055d183420453d6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:01.830362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:01.830654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:01.830936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:01.850980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:01.853080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:01.921204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1056/2105] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:01.928924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:01.929208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:01.929482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:01.951226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:01.952452 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:01.961345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1050/2106] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:01.965847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:01.976012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058cb048-45b7-4659-bc97-c5f339ffb92d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:01.980977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:01.981765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:01.982145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:01.982572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:01.983440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:01.983831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:01.985521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:01.998861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1057/2107] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:55:02.004463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.014431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e26c2-4ec8-47a3-a881-abcd379c2390 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.019245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.020787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.021049 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.021330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.022037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.023991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.051760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1051/2108] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:02 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:02.055805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.066091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecdf2e-8d4a-42a0-933e-a36e24eb492a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.101616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:02.136163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.136863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.137146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.137422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.138115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.140079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.162203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1058/2109] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:02 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:02.166395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.177985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc9243-df4d-49fb-b16b-2142f15223de admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.182746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.183459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.183743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.184019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.184756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.186348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.198149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1052/2110] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:02.202188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.213563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d1433-d9eb-468a-942b-ccee84301e97 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.218251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad700a40-924a-4a5c-8e9f-e4e94c9a468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.218919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad700a40-924a-4a5c-8e9f-e4e94c9a468a admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.219232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad700a40-924a-4a5c-8e9f-e4e94c9a468a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.219508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad700a40-924a-4a5c-8e9f-e4e94c9a468a admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.220237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad700a40-924a-4a5c-8e9f-e4e94c9a468a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.221847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad700a40-924a-4a5c-8e9f-e4e94c9a468a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.232890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1059/2111] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:02.237132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.249280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8242b-f7da-43f9-baa2-a5c4b1942c44 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.254792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.255704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.256070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.256255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] PATH_INFO: `/v3/projects/0b8116f19bab4a3ca51ad9fbd2b9512f/users/5eb23e564ba34c19983b2faef5cf715b/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.273567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8116f19bab4a3ca51ad9fbd2b9512f, user_id=5eb23e564ba34c19983b2faef5cf715b, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.276457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.291647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1053/2112] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:02 2022] PUT /identity/v3/projects/0b8116f19bab4a3ca51ad9fbd2b9512f/users/5eb23e564ba34c19983b2faef5cf715b/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:02.302608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.302899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.303230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.331276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] MFA Rules not processed for user `5eb23e564ba34c19983b2faef5cf715b`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:02.332597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:02.402784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1060/2113] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:02.407353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.471132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a10c-e5e4-463b-9ca5-ecf004f05f5b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.476105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.476777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.477061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.477337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.478041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.480003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.509054 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1054/2114] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:02 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:02.513225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.524941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dcb98-0ca1-411b-96be-19eadd36a52a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.591910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.592585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.592865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.593135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.593828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.595809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.619745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1061/2115] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:02 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:02.624465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.636742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566adec-edfc-40b8-bdc6-f996e923982a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.642556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.643396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.643669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.643946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.644724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.646330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.657715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1055/2116] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:02.662479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.673355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc624c0-662a-4f80-9180-1fedf8323c43 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.680850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.680850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.680850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.680850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] PATH_INFO: `/v3/projects/ce4ef60040634b7a8c36fc8cd08563f8/users/bc38cbcbd1554777a1c23ccbbb846a9b/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.697951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4ef60040634b7a8c36fc8cd08563f8, user_id=bc38cbcbd1554777a1c23ccbbb846a9b, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.700812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.716346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1062/2117] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:02 2022] PUT /identity/v3/projects/ce4ef60040634b7a8c36fc8cd08563f8/users/bc38cbcbd1554777a1c23ccbbb846a9b/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:02.720330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.732002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783f28-72a0-4128-995b-229bb8f73880 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.757286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:02.797389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e64dc-6617-4686-8e9d-10e5a8f34c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.798142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e64dc-6617-4686-8e9d-10e5a8f34c20 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.798423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e64dc-6617-4686-8e9d-10e5a8f34c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.798697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e64dc-6617-4686-8e9d-10e5a8f34c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.799461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e64dc-6617-4686-8e9d-10e5a8f34c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.801620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e64dc-6617-4686-8e9d-10e5a8f34c20 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.812224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1056/2118] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:02.816234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.822208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:02.830291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c4bf10-39e4-440c-b0be-b35596b59389 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:02.836543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:02.837445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.837726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.838052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] PATH_INFO: `/v3/projects/ce4ef60040634b7a8c36fc8cd08563f8/users/bc38cbcbd1554777a1c23ccbbb846a9b/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.840165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4ef60040634b7a8c36fc8cd08563f8, user_id=bc38cbcbd1554777a1c23ccbbb846a9b, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:02.842837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:02.858413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1063/2119] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:02 2022] PUT /identity/v3/projects/ce4ef60040634b7a8c36fc8cd08563f8/users/bc38cbcbd1554777a1c23ccbbb846a9b/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:02.871427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:02.871724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:02.872008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:02.899491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] MFA Rules not processed for user `bc38cbcbd1554777a1c23ccbbb846a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:02.900712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:02.971301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1057/2120] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:02 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:02.982562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:02.984303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:03.040479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2443f97-7001-4d37-ab75-c93612e5bcb1 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:03.044060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaeed7-7c39-485e-9014-614e64711f60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:03.044686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaeed7-7c39-485e-9014-614e64711f60 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:03.044964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaeed7-7c39-485e-9014-614e64711f60 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:03.045236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaeed7-7c39-485e-9014-614e64711f60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:03.054399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aaeed7-7c39-485e-9014-614e64711f60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:03.055857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aaeed7-7c39-485e-9014-614e64711f60 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:03.065094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1064/2121] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:04.762963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:04.852931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a71da8-672c-4ada-bc16-f90d8fb09489 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:04.859699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:04.862665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:04.862665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:04.862665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] PATH_INFO: `/v3/users/5eb23e564ba34c19983b2faef5cf715b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:04.863580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb23e564ba34c19983b2faef5cf715b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:04.865696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:04.933124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb23e564ba34c19983b2faef5cf715b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:04.934740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:04.958128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb23e564ba34c19983b2faef5cf715b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:04.968294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb23e564ba34c19983b2faef5cf715b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:04.978998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb23e564ba34c19983b2faef5cf715b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:04.983308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb23e564ba34c19983b2faef5cf715b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:04.992887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1058/2122] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:04 2022] DELETE /identity/v3/users/5eb23e564ba34c19983b2faef5cf715b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:04.998265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:04.998763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:04.998869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:05.023887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:05.025051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:05.106875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1065/2123] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:05.117402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:05.118898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:05.182154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48333c35-7566-4e29-9f98-82190f97bba3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:05.186397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:05.187086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:05.187383 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:05.187664 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:05.198597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:05.200291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:05.211676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1059/2124] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:05.493537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:05.493537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:05.508355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d8039-7d18-4a6b-ac94-f87af5d925a7 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:05.513535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:05.514400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:05.514718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:05.514943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] PATH_INFO: `/v3/projects/0b8116f19bab4a3ca51ad9fbd2b9512f` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:05.516238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8116f19bab4a3ca51ad9fbd2b9512f)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:05.518874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:05.579517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8116f19bab4a3ca51ad9fbd2b9512f', 'request_id': 'req-52ed1291-a8fe-4c6c-a3a4-60854327caee'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.587184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8116f19bab4a3ca51ad9fbd2b9512f', 'request_id': 'req-52ed1291-a8fe-4c6c-a3a4-60854327caee'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.593248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8116f19bab4a3ca51ad9fbd2b9512f', 'request_id': 'req-52ed1291-a8fe-4c6c-a3a4-60854327caee'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.607183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] The token cache is being invalidate because project 0b8116f19bab4a3ca51ad9fbd2b9512f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:05.607491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.608103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.609668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1066/2125] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:05 2022] DELETE /identity/v3/projects/0b8116f19bab4a3ca51ad9fbd2b9512f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:05.614488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:05.624411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b8cef-2153-4bac-a601-25f8ce56ae47 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:05.685407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:05.686287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:05.686568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:05.686845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] PATH_INFO: `/v3/users/bc38cbcbd1554777a1c23ccbbb846a9b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:05.688088 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc38cbcbd1554777a1c23ccbbb846a9b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:05.690043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:05.737301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc38cbcbd1554777a1c23ccbbb846a9b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.750796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc38cbcbd1554777a1c23ccbbb846a9b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.755445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:05.758572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc38cbcbd1554777a1c23ccbbb846a9b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.766459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc38cbcbd1554777a1c23ccbbb846a9b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.767205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b211683e-3685-49c6-be91-976fef7e2689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc38cbcbd1554777a1c23ccbbb846a9b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:05.774250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1060/2126] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:05 2022] DELETE /identity/v3/users/bc38cbcbd1554777a1c23ccbbb846a9b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:06.071259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.072808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:06.093143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed1291-a8fe-4c6c-a3a4-60854327caee admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.108026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:06.157253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.158117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.158400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.158673 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] PATH_INFO: `/v3/projects/ce4ef60040634b7a8c36fc8cd08563f8` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.159935 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4ef60040634b7a8c36fc8cd08563f8)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.162371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.212378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4ef60040634b7a8c36fc8cd08563f8', 'request_id': 'req-b3a9c117-2526-4f46-83e3-025641d5512c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:06.218697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4ef60040634b7a8c36fc8cd08563f8', 'request_id': 'req-b3a9c117-2526-4f46-83e3-025641d5512c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:06.225975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4ef60040634b7a8c36fc8cd08563f8', 'request_id': 'req-b3a9c117-2526-4f46-83e3-025641d5512c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:06.241269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] The token cache is being invalidate because project ce4ef60040634b7a8c36fc8cd08563f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:06.241576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:06.242398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-b3a9c117-2526-4f46-83e3-025641d5512c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:06.244100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1067/2127] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:06 2022] DELETE /identity/v3/projects/ce4ef60040634b7a8c36fc8cd08563f8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:06.249331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.249697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.249984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.268361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:06.270287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:06.331170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1061/2128] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:06.339806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.340096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.340404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.360278 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:06.361526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:06.371879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1068/2129] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:06.376140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.388307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a95bb5-c7b1-48e1-ab1b-e93250df3170 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.392339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.393077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.393363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.393637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.394986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:06.395903 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.397507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.409343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1062/2130] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:55:06.414377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.423530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42040427-9fc9-491b-9248-ef5dfdfc5a11 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.428534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.429214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.429491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.429762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.430469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.432428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.461233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1069/2131] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:06.464392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.474957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5773570-ffc5-4028-8930-9ecec6396ad7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.533688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.534367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.534653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.534932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.535757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.537712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.563022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1063/2132] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:06.564224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.576212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800fcb4-794d-48d1-a3ab-7e250f8d5f7e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.582622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.583349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.583634 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.583914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.584657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.586284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.599313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1070/2133] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:06.603603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.614607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17033708-2694-409b-8eed-71cc9afed947 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.619417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c953c8-2cac-4045-9677-e7a8b70ce097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.620364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c953c8-2cac-4045-9677-e7a8b70ce097 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.620612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c953c8-2cac-4045-9677-e7a8b70ce097 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.620896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c953c8-2cac-4045-9677-e7a8b70ce097 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.621627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c953c8-2cac-4045-9677-e7a8b70ce097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.623603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c953c8-2cac-4045-9677-e7a8b70ce097 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.635369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1064/2134] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:06.639640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.651531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07178f-3602-4297-acab-05c39c28b8fa admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.659587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.661212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.661716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.662221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] PATH_INFO: `/v3/projects/f4a075a7ba224b22bb25ee070268aec2/users/b3a9614d3703460184e19099c93954f7/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.681350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a075a7ba224b22bb25ee070268aec2, user_id=b3a9614d3703460184e19099c93954f7, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.684149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.698882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1071/2135] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:06 2022] PUT /identity/v3/projects/f4a075a7ba224b22bb25ee070268aec2/users/b3a9614d3703460184e19099c93954f7/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:06.710618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.710904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.711298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.738889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] MFA Rules not processed for user `b3a9614d3703460184e19099c93954f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:06.740149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:06.782337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:06.809890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1065/2136] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:06.814408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.859876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:06.879992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06af27-4b8b-4114-ad77-42bde6df22cc admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:06.884783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:06.885447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:06.885770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:06.886048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:06.886755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:06.888786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:06.916015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1072/2137] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:06.920133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:06.931911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2b8e3-1578-47eb-88de-64f511b7e4a4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.001062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.001749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.002036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.002461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.003255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.005927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.032161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1066/2138] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:06 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:07.036132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:07.047738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fc2df-e434-42df-ba21-01701f428a85 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.053706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.054373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.054649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.054920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.055917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.058227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.069835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1073/2139] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:07.073891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:07.084759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253ea4b-e4d7-497b-abfc-8e2a6829c51f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.091341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.092240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.092522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.092794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] PATH_INFO: `/v3/projects/0ce58ef1af4a4bd2b890781034974ce7/users/2c561e6b9811445e9224f4f02aa1af14/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.108871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce58ef1af4a4bd2b890781034974ce7, user_id=2c561e6b9811445e9224f4f02aa1af14, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.111547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.127148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1067/2140] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:07 2022] PUT /identity/v3/projects/0ce58ef1af4a4bd2b890781034974ce7/users/2c561e6b9811445e9224f4f02aa1af14/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:07.131190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:07.142637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e78ce-a47a-4a7b-8fa5-239328630468 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.199100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4bc921-2ce8-4e60-a09b-f9918407db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.199771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bc921-2ce8-4e60-a09b-f9918407db9d admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.200067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bc921-2ce8-4e60-a09b-f9918407db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.200323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4bc921-2ce8-4e60-a09b-f9918407db9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.201049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bc921-2ce8-4e60-a09b-f9918407db9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.202657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4bc921-2ce8-4e60-a09b-f9918407db9d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.212853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1074/2141] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:07.217096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:07.228386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bee2cc-e29f-4a0a-85bc-e3588514718f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.233062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.233948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.234223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.234490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] PATH_INFO: `/v3/projects/0ce58ef1af4a4bd2b890781034974ce7/users/2c561e6b9811445e9224f4f02aa1af14/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.236617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce58ef1af4a4bd2b890781034974ce7, user_id=2c561e6b9811445e9224f4f02aa1af14, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.240170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.251940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1068/2142] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:07 2022] PUT /identity/v3/projects/0ce58ef1af4a4bd2b890781034974ce7/users/2c561e6b9811445e9224f4f02aa1af14/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:07.263229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.263621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.263888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.292738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] MFA Rules not processed for user `2c561e6b9811445e9224f4f02aa1af14`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:07.293958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:07.367526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1075/2143] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:07.378741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:07.380518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:07.442558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34a5f4-c5ea-4aa2-9b96-dc6ff82eb75c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.446740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.447458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.447726 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.448024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.459616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.461063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.471415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1069/2144] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:07 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:55:07.551502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:07.562694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1197117-22cc-44c9-9aa5-fc8608a6d20a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:07.566495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73926419-6a31-4c27-bb98-31ec78dbeb21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:07.567152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73926419-6a31-4c27-bb98-31ec78dbeb21 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:07.567437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73926419-6a31-4c27-bb98-31ec78dbeb21 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:07.567716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73926419-6a31-4c27-bb98-31ec78dbeb21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:07.607135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:07.632102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73926419-6a31-4c27-bb98-31ec78dbeb21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:07.634188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73926419-6a31-4c27-bb98-31ec78dbeb21 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:07.643723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1076/2145] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:17.001993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:17.042905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:17.065715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec46b6-cca0-4be2-a90b-1eb635b0b675 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:17.070271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:17.071207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.071492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.071769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] PATH_INFO: `/v3/users/b3a9614d3703460184e19099c93954f7` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.073003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a9614d3703460184e19099c93954f7)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:17.074960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:17.124344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a9614d3703460184e19099c93954f7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.136296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a9614d3703460184e19099c93954f7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.143194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a9614d3703460184e19099c93954f7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.151178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a9614d3703460184e19099c93954f7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.151867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a9614d3703460184e19099c93954f7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.157950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1070/2146] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:17 2022] DELETE /identity/v3/users/b3a9614d3703460184e19099c93954f7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:17.162561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7266c-455e-4b56-93d3-a41f486fac64 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.163044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7266c-455e-4b56-93d3-a41f486fac64 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.163182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7266c-455e-4b56-93d3-a41f486fac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.183395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3ff7266c-455e-4b56-93d3-a41f486fac64 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:17.184596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3ff7266c-455e-4b56-93d3-a41f486fac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:17.245928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1077/2147] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:17.255508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:17.256932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:17.314483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481efe-13ef-40e1-875f-6e29d1254bd3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:17.318221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:17.318813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.319124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.319405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.328659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:17.330204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:17.341554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1071/2148] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:55:17.443817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.444178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.446359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.464973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:17.466494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:17.536227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1078/2149] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:55:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:17.587083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:17.590377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:17.699696 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:17.701245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:17.702001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:17.709214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897e413-eb05-460a-a44d-cc76e5f96ec0 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:17.717829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:17.718665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.719423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.719792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.733590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed764919-62a9-4dab-a6fe-fec31e71c5a8 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:17.739152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:17.740380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.740940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.741292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] PATH_INFO: `/v3/projects/f4a075a7ba224b22bb25ee070268aec2` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.750815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a075a7ba224b22bb25ee070268aec2)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:17.752899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:17.753811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:17.759948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:17.794392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1072/2150] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:55:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:55:17.828112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a075a7ba224b22bb25ee070268aec2', 'request_id': 'req-fa3ded1e-bcda-4189-9891-fa0b60491f2d'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.839514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a075a7ba224b22bb25ee070268aec2', 'request_id': 'req-fa3ded1e-bcda-4189-9891-fa0b60491f2d'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.846435 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a075a7ba224b22bb25ee070268aec2', 'request_id': 'req-fa3ded1e-bcda-4189-9891-fa0b60491f2d'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.867653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] The token cache is being invalidate because project f4a075a7ba224b22bb25ee070268aec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:17.868077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.869047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:17.872434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1079/2151] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:17 2022] DELETE /identity/v3/projects/f4a075a7ba224b22bb25ee070268aec2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:17.877838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:17.890992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009666a9-b17b-4176-8087-d5d34208b3ae service swift] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:17.973193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:17.974070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:17.974357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:17.974630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] PATH_INFO: `/v3/users/2c561e6b9811445e9224f4f02aa1af14` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:17.975981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c561e6b9811445e9224f4f02aa1af14)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:17.977949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:18.034725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c561e6b9811445e9224f4f02aa1af14'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.052440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c561e6b9811445e9224f4f02aa1af14'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.060845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c561e6b9811445e9224f4f02aa1af14'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.069223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c561e6b9811445e9224f4f02aa1af14'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.070483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-9a186c5f-c4be-429e-8f87-c28d1d57d033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c561e6b9811445e9224f4f02aa1af14'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.080197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1073/2152] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:17 2022] DELETE /identity/v3/users/2c561e6b9811445e9224f4f02aa1af14 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:18.372676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:18.374694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:18.395264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:18.407170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ded1e-bcda-4189-9891-fa0b60491f2d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:18.479657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:18.480530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.480805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.481074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] PATH_INFO: `/v3/projects/0ce58ef1af4a4bd2b890781034974ce7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.482294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce58ef1af4a4bd2b890781034974ce7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:18.484346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:18.536308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce58ef1af4a4bd2b890781034974ce7', 'request_id': 'req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.543404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce58ef1af4a4bd2b890781034974ce7', 'request_id': 'req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.550830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce58ef1af4a4bd2b890781034974ce7', 'request_id': 'req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.575273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] The token cache is being invalidate because project 0ce58ef1af4a4bd2b890781034974ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:18.576206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.577107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7de1f72b-5486-44cf-a15c-288dfdfa1cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:18.580244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1080/2153] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:18 2022] DELETE /identity/v3/projects/0ce58ef1af4a4bd2b890781034974ce7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:18.590051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.590580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.591000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.613135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:18.614736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:18.683766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1074/2154] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:18.691529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.691817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.692093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.714197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:18.715684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:18.725903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1081/2155] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:18.732208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:18.743666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57fd3d-785a-41c9-9a7b-b9afb1da5d58 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:18.749194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:18.750108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.750398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.750681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.751502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:18.751826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:18.753497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:18.757124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:18.771108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1075/2156] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:55:18.778544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:18.790863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba1e8e-7aa2-4b1e-9e5b-d68ccdf6d767 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:18.796066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:18.796724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.797004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.797277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.797984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:18.799936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:18.832447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1082/2157] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:18 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:18.838227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:18.852381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03072473-bb78-4d4f-b551-7796ee8b3828 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:18.937911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:18.938695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.939053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.939424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.940341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:18.942355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:18.965499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1076/2158] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:18 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:18.970092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:18.984299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeba74-54d7-4dc9-9a0d-994dcdaa2616 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:18.990094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:18.991142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:18.991931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:18.992899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:18.993674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:18.996079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.009933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1083/2159] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:19.013845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.028398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0632df-8925-4eaf-b500-32b266d9a36e admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.036021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63936f57-b1a5-4c8b-bcca-aee4f751cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.037187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63936f57-b1a5-4c8b-bcca-aee4f751cbfa admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.037735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63936f57-b1a5-4c8b-bcca-aee4f751cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.038203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63936f57-b1a5-4c8b-bcca-aee4f751cbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.040176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63936f57-b1a5-4c8b-bcca-aee4f751cbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.043283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63936f57-b1a5-4c8b-bcca-aee4f751cbfa admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.062969 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1077/2160] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:19.064427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.076146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20bcb0-7f2d-4f3f-8b3f-d12edf08af80 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.093671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.094861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.095485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.095711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] PATH_INFO: `/v3/projects/53f62e013b084abea4a99495b63edfdd/users/f233938146ea43b9b9875bb1e7b44caa/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.118331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f62e013b084abea4a99495b63edfdd, user_id=f233938146ea43b9b9875bb1e7b44caa, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.121425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.151939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1084/2161] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:19 2022] PUT /identity/v3/projects/53f62e013b084abea4a99495b63edfdd/users/f233938146ea43b9b9875bb1e7b44caa/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:19.153816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.154117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.154396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.287243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] MFA Rules not processed for user `f233938146ea43b9b9875bb1e7b44caa`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:19.291231 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:19.400884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1078/2162] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:19.405315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.453675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:19.502976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78564c-7140-45de-8d83-c2089b9dd866 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.510535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.511446 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.511724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.512254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.513018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.515293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.551132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1085/2163] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:19 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:19.553750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.568474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10d6c9-dccc-4653-bcd1-7ad3ce83941d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.645396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.646081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.646364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.646639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.647374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.650558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.659729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:19.676206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1079/2164] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:19 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:19.680214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.691325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33882e99-87a7-44cc-ae57-ebee46dbe3cb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.696864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.697536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.697916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.698294 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.699303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.700881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.717172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1086/2165] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:19.721329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.733275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ee7c6-1343-4cb5-93fe-cf1b5ad267aa admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.738759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.739951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.740320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.740645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] PATH_INFO: `/v3/projects/c4f1097d2c79498caa6f6ebe51f0d2cc/users/b375e3e9e7664eb08dd9f6fb4af6dc90/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.758973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f1097d2c79498caa6f6ebe51f0d2cc, user_id=b375e3e9e7664eb08dd9f6fb4af6dc90, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.761765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.777714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1080/2166] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:19 2022] PUT /identity/v3/projects/c4f1097d2c79498caa6f6ebe51f0d2cc/users/b375e3e9e7664eb08dd9f6fb4af6dc90/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:19.781684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.793242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b0d86-aeb9-4765-a341-48e25147675e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.856197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d956d-374f-49ce-a07f-60c9281d99b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.856884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d956d-374f-49ce-a07f-60c9281d99b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.857176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d956d-374f-49ce-a07f-60c9281d99b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.857723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d956d-374f-49ce-a07f-60c9281d99b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.858530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d956d-374f-49ce-a07f-60c9281d99b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.861400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d956d-374f-49ce-a07f-60c9281d99b1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.875519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1087/2167] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:19.886453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:19.901457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1e4d-4ba4-4d8b-abf1-d36cf9df96bb admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:19.908888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:19.910523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.911132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.911535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] PATH_INFO: `/v3/projects/c4f1097d2c79498caa6f6ebe51f0d2cc/users/b375e3e9e7664eb08dd9f6fb4af6dc90/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.915626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f1097d2c79498caa6f6ebe51f0d2cc, user_id=b375e3e9e7664eb08dd9f6fb4af6dc90, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:19.919958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:19.949002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1081/2168] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:19 2022] PUT /identity/v3/projects/c4f1097d2c79498caa6f6ebe51f0d2cc/users/b375e3e9e7664eb08dd9f6fb4af6dc90/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:19.951084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:19.951676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:19.952057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:19.984381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] MFA Rules not processed for user `b375e3e9e7664eb08dd9f6fb4af6dc90`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:19.985558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:20.064997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1088/2169] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:20.082507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:20.084503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:20.176277 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23ced3-d521-4ec0-b351-b0d512f1c0e9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:20.183131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:20.184249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:20.184763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:20.185387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:20.199470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:20.202014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:20.216309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1082/2170] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:55:20.323245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:20.334070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c140893-c073-4015-ac62-4ea1bee107ff None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:20.339332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c774c-d681-4aaf-aedf-49767b69cb26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:20.339989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c774c-d681-4aaf-aedf-49767b69cb26 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:20.340273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c774c-d681-4aaf-aedf-49767b69cb26 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:20.340552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c774c-d681-4aaf-aedf-49767b69cb26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:20.373600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:20.467020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c774c-d681-4aaf-aedf-49767b69cb26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:20.467020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193c774c-d681-4aaf-aedf-49767b69cb26 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:20.472493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1089/2171] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:25.409731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:25.531609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd366c4-680c-4f0d-ad4e-fc1635c1eeb0 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:25.538303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:25.539945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:25.540313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:25.540658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] PATH_INFO: `/v3/users/f233938146ea43b9b9875bb1e7b44caa` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:25.542149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f233938146ea43b9b9875bb1e7b44caa)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:25.545872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:25.694013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f233938146ea43b9b9875bb1e7b44caa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:25.716422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:25.722408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f233938146ea43b9b9875bb1e7b44caa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:25.778145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f233938146ea43b9b9875bb1e7b44caa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:25.817730 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f233938146ea43b9b9875bb1e7b44caa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:25.818995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f233938146ea43b9b9875bb1e7b44caa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:25.837430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1083/2172] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:25 2022] DELETE /identity/v3/users/f233938146ea43b9b9875bb1e7b44caa => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:25.846358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:25.846358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:25.846358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:25.873678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:25.875394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:26.048546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1090/2173] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:26.060114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:26.062009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:26.189107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703da026-4531-487c-915d-f496da32ad1a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:26.193173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:26.193838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:26.194176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:26.194459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:26.207366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:26.209035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:26.221988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1084/2174] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:26.598680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:26.600764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:26.628416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe680819-221e-4bdd-8e7f-998305054556 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:26.638406 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:26.640054 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:26.640341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:26.640617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] PATH_INFO: `/v3/projects/53f62e013b084abea4a99495b63edfdd` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:26.642072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f62e013b084abea4a99495b63edfdd)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:26.644211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:26.775770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f62e013b084abea4a99495b63edfdd', 'request_id': 'req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.787967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f62e013b084abea4a99495b63edfdd', 'request_id': 'req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.798803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f62e013b084abea4a99495b63edfdd', 'request_id': 'req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.815902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] The token cache is being invalidate because project 53f62e013b084abea4a99495b63edfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:26.816241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.816892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.818610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1091/2175] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:26 2022] DELETE /identity/v3/projects/53f62e013b084abea4a99495b63edfdd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:26.823389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:26.844032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095a236-3ec0-4600-9ac3-949b1bdb25dd service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:26.903087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:26.903983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:26.904268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:26.904560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] PATH_INFO: `/v3/users/b375e3e9e7664eb08dd9f6fb4af6dc90` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:26.905755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b375e3e9e7664eb08dd9f6fb4af6dc90)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:26.907808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:26.963008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b375e3e9e7664eb08dd9f6fb4af6dc90'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.977682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b375e3e9e7664eb08dd9f6fb4af6dc90'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.985194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b375e3e9e7664eb08dd9f6fb4af6dc90'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.992977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b375e3e9e7664eb08dd9f6fb4af6dc90'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.993640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5e6dd2a2-5539-4d2e-a024-6777a291eec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b375e3e9e7664eb08dd9f6fb4af6dc90'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:26.996130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:27.001287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1085/2176] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:26 2022] DELETE /identity/v3/users/b375e3e9e7664eb08dd9f6fb4af6dc90 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:27.390909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:27.392414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:27.424493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3e666-b2ff-4d08-a3b0-5972dc19be6f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:27.474602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:27.490781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:27.491671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.491952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.492330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] PATH_INFO: `/v3/projects/c4f1097d2c79498caa6f6ebe51f0d2cc` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.494315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f1097d2c79498caa6f6ebe51f0d2cc)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:27.496965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:27.551795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f1097d2c79498caa6f6ebe51f0d2cc', 'request_id': 'req-ef043b2e-110b-491b-b4a2-96671259489b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:27.559149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f1097d2c79498caa6f6ebe51f0d2cc', 'request_id': 'req-ef043b2e-110b-491b-b4a2-96671259489b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:27.565845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f1097d2c79498caa6f6ebe51f0d2cc', 'request_id': 'req-ef043b2e-110b-491b-b4a2-96671259489b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:27.579405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] The token cache is being invalidate because project c4f1097d2c79498caa6f6ebe51f0d2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:27.579702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:27.580416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ef043b2e-110b-491b-b4a2-96671259489b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:27.582031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1092/2177] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:27 2022] DELETE /identity/v3/projects/c4f1097d2c79498caa6f6ebe51f0d2cc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:27.587880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.588259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.588633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.615217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:27.616543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:27.684217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1086/2178] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:27.692210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.692534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.692656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.713550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:27.714928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:27.722936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1093/2179] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:27.729882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:27.740187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc922ea7-8caf-4a82-beb4-cdc5d083c2fe None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:27.745556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:27.746226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.746503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.746774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.747621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:27.747859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:27.749462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:27.762428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1087/2180] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:55:27.766631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:27.778693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477b7d0-4700-40e8-b5d0-db9f0a764424 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:27.783286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:27.783939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.784216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.784490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.785188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:27.787604 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:27.821250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1094/2181] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:27 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:27.825394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:27.836442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942cae9-e96f-4d1a-8f4a-fc96f099ad63 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:27.934563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:27.935938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.936125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.936419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.937668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:27.940283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:27.973031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1088/2182] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:27 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:27.974259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:27.986713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be354fe5-c49f-4810-8c90-1a2f8c07d1a2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:27.991863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:27.993203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:27.993547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:27.993853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:27.994741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:27.997261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.013411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1095/2183] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:28.017860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.029463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750553a9-0048-4143-904a-4b27a84e7631 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.034267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754f0d4-ee3f-4592-9729-b9ac07b2f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.034920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f0d4-ee3f-4592-9729-b9ac07b2f070 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.035243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f0d4-ee3f-4592-9729-b9ac07b2f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.035514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754f0d4-ee3f-4592-9729-b9ac07b2f070 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.036243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754f0d4-ee3f-4592-9729-b9ac07b2f070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.037860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754f0d4-ee3f-4592-9729-b9ac07b2f070 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.051096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1089/2184] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:28.053177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.063977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b69407-2436-49bd-b46b-d42c327908e7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.069368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.070881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.071288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.071579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] PATH_INFO: `/v3/projects/cae582fadc064b8d8fb0c352a7970f5b/users/07fe79c99b024f07a8c9e46f5c0f0c72/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.093908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae582fadc064b8d8fb0c352a7970f5b, user_id=07fe79c99b024f07a8c9e46f5c0f0c72, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.096785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.110633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1096/2185] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:28 2022] PUT /identity/v3/projects/cae582fadc064b8d8fb0c352a7970f5b/users/07fe79c99b024f07a8c9e46f5c0f0c72/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:28.121640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.121860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.122142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.154851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] MFA Rules not processed for user `07fe79c99b024f07a8c9e46f5c0f0c72`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:28.156126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:28.215900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:28.228434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1090/2186] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:28.233295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.303944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bc2b7-4e5d-4a98-a1fc-66deab71f8be admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.309194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.309920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.310211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.310488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.311217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.313145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.334393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:28.345717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1097/2187] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:28 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:28.347105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.359174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fef26-c6c9-4266-9526-994a817165e5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.434714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.435428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.435866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.435866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.436729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.438704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.471130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1091/2188] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:28 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:28.475233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.489274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04f246-6c05-4281-aedd-925f3c334d36 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.496028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.496732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.496998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.497276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.498018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.499731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.513676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1098/2189] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:28.518543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.533248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4422b-7ffc-45ce-8490-4cea791bcf97 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.541138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.544376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.544376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.544376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] PATH_INFO: `/v3/projects/696db42baaba48688c1d45f9312fa6e4/users/41c53a2dd82146a0a8aeb6136428ffb3/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.567583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696db42baaba48688c1d45f9312fa6e4, user_id=41c53a2dd82146a0a8aeb6136428ffb3, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.570396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.589656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1092/2190] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:28 2022] PUT /identity/v3/projects/696db42baaba48688c1d45f9312fa6e4/users/41c53a2dd82146a0a8aeb6136428ffb3/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:28.593652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.607102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea40ab-f708-4f2a-bee8-33adbeb9e6f4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.668907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5cac20-3c86-4406-8113-8644a7e12d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.669503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cac20-3c86-4406-8113-8644a7e12d65 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.669786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cac20-3c86-4406-8113-8644a7e12d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.670065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5cac20-3c86-4406-8113-8644a7e12d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.670805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cac20-3c86-4406-8113-8644a7e12d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.672481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5cac20-3c86-4406-8113-8644a7e12d65 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.684758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1099/2191] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:28.688883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.701940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6b4b-b7f6-4531-9973-bd3567dcc8e8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.706691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.707634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.707919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.708198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] PATH_INFO: `/v3/projects/696db42baaba48688c1d45f9312fa6e4/users/41c53a2dd82146a0a8aeb6136428ffb3/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.710205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=696db42baaba48688c1d45f9312fa6e4, user_id=41c53a2dd82146a0a8aeb6136428ffb3, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:28.713475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:28.724185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1093/2192] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:28 2022] PUT /identity/v3/projects/696db42baaba48688c1d45f9312fa6e4/users/41c53a2dd82146a0a8aeb6136428ffb3/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:28.735354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.735658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.735912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:28.769224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] MFA Rules not processed for user `41c53a2dd82146a0a8aeb6136428ffb3`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:28.770622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:28.855697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1100/2193] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:28.867508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:28.869395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:28.938378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a698f-97a5-4950-90b0-9d34a5b71d2d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:28.942442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d8008-9c22-4213-9ad2-e30a8d9b2fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:28.943123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8008-9c22-4213-9ad2-e30a8d9b2fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:28.943408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8008-9c22-4213-9ad2-e30a8d9b2fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:28.943719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d8008-9c22-4213-9ad2-e30a8d9b2fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:29.011401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8008-9c22-4213-9ad2-e30a8d9b2fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:29.012872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d8008-9c22-4213-9ad2-e30a8d9b2fa7 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:29.023253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1094/2194] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:29.058737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:29.060376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:29.188745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df662542-28ff-48f7-bcb0-c18e080387e9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:29.203732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:29.207001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:29.207422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:29.207746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] PATH_INFO: `/v3/users/cf13b31229794b81b9e6846c9bae8625` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:29.209734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf13b31229794b81b9e6846c9bae8625)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:29.220576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:29.235862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:29.351288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf13b31229794b81b9e6846c9bae8625'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:29.380197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf13b31229794b81b9e6846c9bae8625'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:29.396360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf13b31229794b81b9e6846c9bae8625'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:29.398182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf13b31229794b81b9e6846c9bae8625'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:29.399535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf13b31229794b81b9e6846c9bae8625'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:29.418940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1101/2195] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:29 2022] DELETE /identity/v3/users/cf13b31229794b81b9e6846c9bae8625 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:29.424472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:29.424757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:29.425034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:29.434328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:29.456770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:29.458149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:29.600267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1095/2196] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:29.610685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:29.613560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:29.707426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8b8b5-a131-4358-baf2-16f624a48718 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:29.712948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04122e8-458b-4267-9ad4-961a241ae3e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:29.713934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04122e8-458b-4267-9ad4-961a241ae3e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:29.716311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04122e8-458b-4267-9ad4-961a241ae3e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:29.716311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04122e8-458b-4267-9ad4-961a241ae3e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:29.728623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04122e8-458b-4267-9ad4-961a241ae3e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:29.730188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04122e8-458b-4267-9ad4-961a241ae3e8 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:29.743800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1102/2197] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:30.091194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.092898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:30.110899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc9198-5eac-4c83-9eb5-090e78fb229c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.116227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.117427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.117848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.118127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] PATH_INFO: `/v3/projects/94f303a44f7b4916a06c55e64ed2dc49` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.119378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f303a44f7b4916a06c55e64ed2dc49)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.122626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.174415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f303a44f7b4916a06c55e64ed2dc49', 'request_id': 'req-5efb50ba-d28d-4ea9-862c-86208e20d978'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:30.181435 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f303a44f7b4916a06c55e64ed2dc49', 'request_id': 'req-5efb50ba-d28d-4ea9-862c-86208e20d978'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:30.188872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f303a44f7b4916a06c55e64ed2dc49', 'request_id': 'req-5efb50ba-d28d-4ea9-862c-86208e20d978'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:30.204308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] The token cache is being invalidate because project 94f303a44f7b4916a06c55e64ed2dc49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:30.204637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:30.205478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5efb50ba-d28d-4ea9-862c-86208e20d978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:30.208085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1096/2198] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:30 2022] DELETE /identity/v3/projects/94f303a44f7b4916a06c55e64ed2dc49 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:30.213372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.213672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.213942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.234686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:30.236077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:30.306441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1103/2199] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:30.314338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.314687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.314981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.335999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:30.337964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:30.346296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1097/2200] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:30.350513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.361189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678b489-d532-4e4f-8fa7-f48d17d0663c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.365785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.366444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.366729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.367018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.368030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:30.368344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.370217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.384135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1104/2201] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:55:30.388784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.399097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d7050-3ff1-4079-80b3-64c123389d0e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.404295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.404961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.405455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.405571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.406204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.408310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.437019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1098/2202] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:30 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:30.440858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.453900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7bc0a-f336-4685-855b-b144646a89d7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.475513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:30.530096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.530846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.531223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.531651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.532357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.534648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.559815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1105/2203] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:30 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:30.565144 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.580288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbcb01-7023-4541-be55-5082d0cceb41 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.588044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.588843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.589126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.589405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.590249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.592820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.609012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1099/2204] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:30.614830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.653733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef273-6a01-4071-8b44-9198e7b7c50c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.664843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5207bd-a4ad-4d60-b622-0f4705aa517e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.666161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5207bd-a4ad-4d60-b622-0f4705aa517e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.667101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5207bd-a4ad-4d60-b622-0f4705aa517e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.667582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5207bd-a4ad-4d60-b622-0f4705aa517e admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.668747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5207bd-a4ad-4d60-b622-0f4705aa517e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.670704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5207bd-a4ad-4d60-b622-0f4705aa517e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.690341 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1106/2205] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:30.692058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:30.736759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31978d-b722-4d59-bfd3-4727842ef39d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:30.756223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:30.761020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.761515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.762012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] PATH_INFO: `/v3/projects/8bb3a584e256464d845e6f0f18d9b78e/users/31145ada7a534ff584a502fe0a453090/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.800725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bb3a584e256464d845e6f0f18d9b78e, user_id=31145ada7a534ff584a502fe0a453090, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:30.805326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:30.830825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1100/2206] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:30 2022] PUT /identity/v3/projects/8bb3a584e256464d845e6f0f18d9b78e/users/31145ada7a534ff584a502fe0a453090/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:30.842846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf32c22-4597-49cd-8917-4008584f79bc None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:30.843676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf32c22-4597-49cd-8917-4008584f79bc None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:30.844569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf32c22-4597-49cd-8917-4008584f79bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:30.918354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-bbf32c22-4597-49cd-8917-4008584f79bc None None] MFA Rules not processed for user `31145ada7a534ff584a502fe0a453090`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:30.920946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bbf32c22-4597-49cd-8917-4008584f79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:31.042163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1107/2207] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:31.054127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:31.057296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:31.092956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:31.189448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aed344-0d82-43ff-963d-9786a5d049f9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:31.197056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:31.198528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:31.199163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:31.199809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:31.219267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:31.222511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:31.250593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1101/2208] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:55:34.737307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:34.737853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:34.738270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:34.779658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] MFA Rules not processed for user `1a0d4ce0ac4d47c8a184ea0291461379`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:34.780963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:34.907148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1108/2209] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:38.677900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:38.694545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc06b46-e0e2-4af7-9c55-02565209897d service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:38.714885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:38.715610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:38.717905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:38.718373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:38.726122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:38.832187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:38.834239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:38.862764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1102/2210] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:44.893293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:48.702724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:48.705168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:48.844912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1e922-bbac-46fa-b687-ee1593a91418 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:48.850774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:48.851755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=46 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:48.852396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:48.852859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:48.853212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:48.855761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:48.871183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:48.873357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad4404-2e5a-402c-ad28-29aa038a8e5b service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:48.886897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1109/2211] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:52.583016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:52.586261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:52.719647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4885c-a917-413f-a4ee-2a949185cfb4 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:52.728001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:52.729131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:52.729685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:52.729794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] PATH_INFO: `/v3/users/07fe79c99b024f07a8c9e46f5c0f0c72` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:52.732620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fe79c99b024f07a8c9e46f5c0f0c72)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:52.735529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:52.805752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fe79c99b024f07a8c9e46f5c0f0c72'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:52.822972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fe79c99b024f07a8c9e46f5c0f0c72'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:52.832507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fe79c99b024f07a8c9e46f5c0f0c72'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:52.849093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fe79c99b024f07a8c9e46f5c0f0c72'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:52.851926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fe79c99b024f07a8c9e46f5c0f0c72'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:52.869662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1103/2212] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:52 2022] DELETE /identity/v3/users/07fe79c99b024f07a8c9e46f5c0f0c72 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:52.877272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:52.878881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:52.879685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:52.923574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:52.925022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:53.044643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1110/2213] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:53.058721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:53.060974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:53.191017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1f1d-4f04-4659-8fbf-3f28dcb7a046 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:53.198779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:53.202906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:53.203377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:53.203551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:53.226582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:53.229258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:53.255731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1104/2214] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:55:53.708936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:53.710660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:53.728259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48051770-6af4-42a5-bd00-ceeb55944cfb None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:53.733932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:53.735010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:53.735388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:53.735697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] PATH_INFO: `/v3/projects/cae582fadc064b8d8fb0c352a7970f5b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:53.736947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae582fadc064b8d8fb0c352a7970f5b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:53.738878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:53.847017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae582fadc064b8d8fb0c352a7970f5b', 'request_id': 'req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:53.854970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae582fadc064b8d8fb0c352a7970f5b', 'request_id': 'req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:53.865189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae582fadc064b8d8fb0c352a7970f5b', 'request_id': 'req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:53.879496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] The token cache is being invalidate because project cae582fadc064b8d8fb0c352a7970f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:53.880090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:53.880580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:53.882223 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1111/2215] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:53 2022] DELETE /identity/v3/projects/cae582fadc064b8d8fb0c352a7970f5b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:53.886347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:53.896596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316bca6-36eb-4406-b102-a9e55cf056cb service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:53.917671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:53.962505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:53.963423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:53.963707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:53.963987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] PATH_INFO: `/v3/users/41c53a2dd82146a0a8aeb6136428ffb3` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:53.965213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c53a2dd82146a0a8aeb6136428ffb3)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:53.967269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:54.017172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c53a2dd82146a0a8aeb6136428ffb3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.033175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c53a2dd82146a0a8aeb6136428ffb3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.043351 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c53a2dd82146a0a8aeb6136428ffb3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.050838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c53a2dd82146a0a8aeb6136428ffb3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.051477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-78537e06-e6bc-4052-b83c-256b083dc29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c53a2dd82146a0a8aeb6136428ffb3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.057657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1105/2216] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:53 2022] DELETE /identity/v3/users/41c53a2dd82146a0a8aeb6136428ffb3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:54.351697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:54.354377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:54.368870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624e4f4-80d8-4362-8e78-d2b4a94b56a8 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:54.381415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:54.446823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:54.447550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:54.447859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:54.450480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] PATH_INFO: `/v3/projects/696db42baaba48688c1d45f9312fa6e4` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:54.455968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=696db42baaba48688c1d45f9312fa6e4)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:54.458084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:54.480706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:54.481044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:54.481328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:54.501833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:54.503128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:54.512716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696db42baaba48688c1d45f9312fa6e4', 'request_id': 'req-c23eff36-9024-4641-8d2d-0d3afafb0caf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.520695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696db42baaba48688c1d45f9312fa6e4', 'request_id': 'req-c23eff36-9024-4641-8d2d-0d3afafb0caf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.527517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696db42baaba48688c1d45f9312fa6e4', 'request_id': 'req-c23eff36-9024-4641-8d2d-0d3afafb0caf'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.541405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] The token cache is being invalidate because project 696db42baaba48688c1d45f9312fa6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:54.541695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.542452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:54.544079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1112/2217] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:54 2022] DELETE /identity/v3/projects/696db42baaba48688c1d45f9312fa6e4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:54.569546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1106/2218] 10.222.0.86 () {58 vars in 967 bytes} [Wed Jul 20 21:55:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:54.584225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:54.586176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:54.678417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eff36-9024-4641-8d2d-0d3afafb0caf admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:54.684664 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8fdcd-2757-40f6-93c7-cfe77214b12d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:54.685608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8fdcd-2757-40f6-93c7-cfe77214b12d service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:54.688169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8fdcd-2757-40f6-93c7-cfe77214b12d service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:54.688552 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8fdcd-2757-40f6-93c7-cfe77214b12d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:54.699494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8fdcd-2757-40f6-93c7-cfe77214b12d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:54.701161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8fdcd-2757-40f6-93c7-cfe77214b12d service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:54.711998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1113/2219] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:55:54.893321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:54.894853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:54.966014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:54.989756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffa4f2-cdb7-4df7-bc8f-64ab3c1af2c0 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:54.997245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:54.998617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:54.998996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:54.999527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] PATH_INFO: `/v3/users/31145ada7a534ff584a502fe0a453090` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.001168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31145ada7a534ff584a502fe0a453090)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:55.004617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:55.068179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31145ada7a534ff584a502fe0a453090'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.085802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31145ada7a534ff584a502fe0a453090'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.097618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31145ada7a534ff584a502fe0a453090'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.108338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31145ada7a534ff584a502fe0a453090'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.109251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31145ada7a534ff584a502fe0a453090'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.119707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1107/2220] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:54 2022] DELETE /identity/v3/users/31145ada7a534ff584a502fe0a453090 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:55.126188 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.126735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.126931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.159151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:55.159151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:55.191296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:55.231207 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1114/2221] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:55.243406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:55.245103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:55.306948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41ede-268a-4f82-8c18-7a5df8858668 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:55.310688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e725e-cdc0-4087-8a53-656cb666cf24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:55.311378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e725e-cdc0-4087-8a53-656cb666cf24 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.311657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e725e-cdc0-4087-8a53-656cb666cf24 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.311931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e725e-cdc0-4087-8a53-656cb666cf24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.321766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e725e-cdc0-4087-8a53-656cb666cf24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:55.323309 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e725e-cdc0-4087-8a53-656cb666cf24 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:55.333321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1108/2222] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:55.627553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:55.629131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:55.641777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458080b1-08a4-4dd4-b428-e7ef6d597b5f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:55.647744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:55.649426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.649725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.649990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] PATH_INFO: `/v3/projects/8bb3a584e256464d845e6f0f18d9b78e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.651312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bb3a584e256464d845e6f0f18d9b78e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:55.653384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:55.703554 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bb3a584e256464d845e6f0f18d9b78e', 'request_id': 'req-afccefdb-2bb0-4062-ac56-9f612f19f10b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.711934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bb3a584e256464d845e6f0f18d9b78e', 'request_id': 'req-afccefdb-2bb0-4062-ac56-9f612f19f10b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.717508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bb3a584e256464d845e6f0f18d9b78e', 'request_id': 'req-afccefdb-2bb0-4062-ac56-9f612f19f10b'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.730973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] The token cache is being invalidate because project 8bb3a584e256464d845e6f0f18d9b78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:55.731416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.732040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-afccefdb-2bb0-4062-ac56-9f612f19f10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:55.733710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1115/2223] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:55 2022] DELETE /identity/v3/projects/8bb3a584e256464d845e6f0f18d9b78e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:55.740245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.740524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.740795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.759008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:55.760380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:55.820546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1109/2224] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:55.828004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.828342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.828632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.848445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:55.849658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:55.857815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1116/2225] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:55.862470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:55.873669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b2ebd-15fb-4650-b335-706bfe383cca None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:55.878492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:55.879189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.879479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.879756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.880493 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:55.880805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:55.882909 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:55.885581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:55.898120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1110/2226] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:55:55.902276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:55.912283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6550a-1f03-440c-81f5-c4786597151b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:55.917152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:55.917813 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:55.918094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:55.918367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:55.919070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:55.921005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:55.948524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1117/2227] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:55 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:55.952809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:55.965896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd012c-e6cd-4872-b31f-ddb7b1612144 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:56.028010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:56.028767 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:56.029027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:56.030096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:56.030988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:56.032991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:56.054913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1111/2228] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:55 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:56.058789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:56.069451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e979ec-8eea-4318-a850-05f900168a4a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:56.079793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:56.079793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:56.079793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:56.079793 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:56.080213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:56.080213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:56.095265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1118/2229] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:56.097317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:56.113427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c443b6-0cc9-4e23-8f54-d982cad18e06 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:56.122340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4438d1d-79a3-4b74-8ae1-98241d78ca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:56.123539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438d1d-79a3-4b74-8ae1-98241d78ca9e admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:56.123886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438d1d-79a3-4b74-8ae1-98241d78ca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:56.124096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4438d1d-79a3-4b74-8ae1-98241d78ca9e admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:56.124850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4438d1d-79a3-4b74-8ae1-98241d78ca9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:56.126689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4438d1d-79a3-4b74-8ae1-98241d78ca9e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:56.141148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1112/2230] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:55:56.144679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:56.158092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb743614-53b5-4a1a-bc0e-155bbf532899 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:56.165380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:56.166802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:56.167140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:56.167421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] PATH_INFO: `/v3/projects/c17214008914495b8083b74c6fc9e405/users/030fcf8f4e654d08ac9c7e5dcb3eddd9/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:56.185030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17214008914495b8083b74c6fc9e405, user_id=030fcf8f4e654d08ac9c7e5dcb3eddd9, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:56.187768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:56.203687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1119/2231] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:56 2022] PUT /identity/v3/projects/c17214008914495b8083b74c6fc9e405/users/030fcf8f4e654d08ac9c7e5dcb3eddd9/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:56.208990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f2226-6670-4775-8783-fdc668566bd6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:56.209283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f2226-6670-4775-8783-fdc668566bd6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:56.209571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f2226-6670-4775-8783-fdc668566bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:56.233032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2c8f2226-6670-4775-8783-fdc668566bd6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:56.234256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2c8f2226-6670-4775-8783-fdc668566bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:56.315426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1113/2232] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:56.326279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:56.328033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:56.393129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6492e5-6ad7-497e-ad3d-675f5b090a94 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:56.400228 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fecf8e-aea7-47e9-b186-0475da9fabf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:56.401288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fecf8e-aea7-47e9-b186-0475da9fabf4 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:56.401709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fecf8e-aea7-47e9-b186-0475da9fabf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:56.402146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fecf8e-aea7-47e9-b186-0475da9fabf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:56.413035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:56.417492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fecf8e-aea7-47e9-b186-0475da9fabf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:56.419909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fecf8e-aea7-47e9-b186-0475da9fabf4 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:56.439986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1120/2233] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:55:57.062220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5ab58-6a67-4530-b791-9a2ea4d7eafb None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:57.063413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5ab58-6a67-4530-b791-9a2ea4d7eafb None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:57.064076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5ab58-6a67-4530-b791-9a2ea4d7eafb None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:57.068331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1114/2234] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:55:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:55:57.079099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:57.079099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:57.079099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:57.109800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] MFA Rules not processed for user `06938dbfe3df4bc28f065c82315e4308`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:57.111285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:57.205164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1121/2235] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:55:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:57.206922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5ab58-6a67-4530-b791-9a2ea4d7eafb None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:57.209475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-cdf5ab58-6a67-4530-b791-9a2ea4d7eafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:57.232438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5ab58-6a67-4530-b791-9a2ea4d7eafb None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:57.244146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:57.246092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:57.246479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:57.247028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] PATH_INFO: `/v3/users/9c901aba30d141b39e7189eefc7c1ce1` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:57.248774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c901aba30d141b39e7189eefc7c1ce1)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:57.250846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:57.278050 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:57.279759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:57.302934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7a1a6-82dd-431c-85d3-6ac7e571289e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:57.310173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:57.311104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:57.311302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:57.311583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:57.321747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:57.323577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:57.335021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1122/2236] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:55:57.437614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c901aba30d141b39e7189eefc7c1ce1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:57.463572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c901aba30d141b39e7189eefc7c1ce1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:57.480104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:57.488075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c901aba30d141b39e7189eefc7c1ce1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:57.506118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c901aba30d141b39e7189eefc7c1ce1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:57.507742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-364fa290-fe49-431f-9b85-fdad25425f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c901aba30d141b39e7189eefc7c1ce1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:57.522363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1115/2237] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:55:57 2022] DELETE /identity/v3/users/9c901aba30d141b39e7189eefc7c1ce1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:55:58.422140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:58.456613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa067-5d9b-443f-90ca-cd13251eaf5f service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:58.561451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:58.562324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:58.562804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:58.563270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] PATH_INFO: `/v3/projects/9a43767d64494137af74bbb2955ef55a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:58.564556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a43767d64494137af74bbb2955ef55a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:58.566529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:58.628219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a43767d64494137af74bbb2955ef55a', 'request_id': 'req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:58.635742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a43767d64494137af74bbb2955ef55a', 'request_id': 'req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:58.641641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a43767d64494137af74bbb2955ef55a', 'request_id': 'req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:58.655137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] The token cache is being invalidate because project 9a43767d64494137af74bbb2955ef55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:55:58.655732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:58.656545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0d2411c2-17a3-4b43-8187-ad34ee78d1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:55:58.658173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1123/2238] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:55:58 2022] DELETE /identity/v3/projects/9a43767d64494137af74bbb2955ef55a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:58.663185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:58.663691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:58.663789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:58.682830 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:58.684025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:58.791097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1116/2239] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:58.858135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:58.858476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:58.858722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:58.889433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:58.890670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:58.904110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1124/2240] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:55:58.912008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:58.925322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a9ee0-85e3-48d7-a1cb-c026ae08bcc4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:58.934485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:58.935441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:58.935795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:58.936139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:58.937126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:55:58.937502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:58.939226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:58.962211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1117/2241] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:55:58.968388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:58.973432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:58.980371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba323fe7-bfd8-442b-9bb0-487ecd6514a3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:58.989364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:58.990343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:58.993197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:58.993973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:58.994953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:58.997636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:59.038022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1125/2242] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:55:58 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:55:59.044071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:59.059666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285caab5-786b-4d60-ac1b-3b7ccab990f6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:59.181645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:59.181871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:59.181958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:59.182237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:59.182956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:59.185458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:59.214110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1118/2243] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:55:59 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:55:59.218266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:59.230024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca720b-80d4-431e-9361-714f83fc4dbd admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:59.236304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:59.237033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:59.237369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:59.237650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:59.238441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:59.240235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:59.256922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1126/2244] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:59.258036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:59.271272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3886acd-cd18-482a-8900-09d8383a48c3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:59.278131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e50249-b420-4956-bcd1-2e802b30ecd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:59.278826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e50249-b420-4956-bcd1-2e802b30ecd2 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:59.279139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e50249-b420-4956-bcd1-2e802b30ecd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:59.279423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e50249-b420-4956-bcd1-2e802b30ecd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:59.280178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e50249-b420-4956-bcd1-2e802b30ecd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:59.281818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e50249-b420-4956-bcd1-2e802b30ecd2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:59.294746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1119/2245] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:55:59.301949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:59.312982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515294c2-2d58-41db-a0cf-3735c4b19c95 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:59.318556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:59.319249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:59.319536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:59.319846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] PATH_INFO: `/v3/projects/b4eb21113d83407db43d77aafcb5a33c/users/c9b96b7a3a2142528dd94d5eaf8de9aa/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:59.341166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4eb21113d83407db43d77aafcb5a33c, user_id=c9b96b7a3a2142528dd94d5eaf8de9aa, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:59.345253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:59.366034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1127/2246] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:55:59 2022] PUT /identity/v3/projects/b4eb21113d83407db43d77aafcb5a33c/users/c9b96b7a3a2142528dd94d5eaf8de9aa/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:55:59.376947 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010867ca-03ab-441f-bf3d-f7a47b576434 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:59.377241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010867ca-03ab-441f-bf3d-f7a47b576434 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:59.377525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010867ca-03ab-441f-bf3d-f7a47b576434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:59.412598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-010867ca-03ab-441f-bf3d-f7a47b576434 None None] MFA Rules not processed for user `c9b96b7a3a2142528dd94d5eaf8de9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:55:59.414495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-010867ca-03ab-441f-bf3d-f7a47b576434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:59.468649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:55:59.502437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1120/2247] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:55:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:55:59.512936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:55:59.515955 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:55:59.586613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7f186-8953-40c4-aa14-521bcb259b78 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:55:59.591671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c673d-3437-45a5-8a84-e492df95614b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:55:59.592457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c673d-3437-45a5-8a84-e492df95614b service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:55:59.592749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c673d-3437-45a5-8a84-e492df95614b service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:55:59.593028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c673d-3437-45a5-8a84-e492df95614b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:55:59.602870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c673d-3437-45a5-8a84-e492df95614b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:55:59.604427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c673d-3437-45a5-8a84-e492df95614b service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:55:59.618395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1128/2248] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:55:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:56:01.350392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:01.351745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:01.351745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:01.402113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] MFA Rules not processed for user `c9b96b7a3a2142528dd94d5eaf8de9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:01.403311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:01.425007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1121/2249] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:56:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:56:02.208087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.208616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.208715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.254518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] MFA Rules not processed for user `030fcf8f4e654d08ac9c7e5dcb3eddd9`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:02.256922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:02.364868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1129/2250] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:56:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:02.371922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:02.373611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:02.504628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886cf44-d6e5-4269-a792-67af662184d4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:02.509880 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:02.510551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.510832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.511148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.511878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:02.513839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:02.549330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1122/2251] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:56:02 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:56:02.554374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:02.575775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec3c26-65a0-4bb9-886a-6e91f1c1f359 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:02.643548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:02.672371 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:02.673929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.673929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.674286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.675740 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:02.679135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:02.733588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1130/2252] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:56:02 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:56:02.737446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:02.751473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8c006-14a9-4dc7-a073-db1e3a9f3720 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:02.758698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:02.759695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.759980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.760259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.761001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:02.762625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:02.775113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1123/2253] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:56:02.780386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:02.799828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a862b-e907-4c03-8eb4-04cdd08899c7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:02.808227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:02.809794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.810290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.810790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] PATH_INFO: `/v3/projects/3ee21db55ee449d7be269c7eaa58b878/users/0a7cd440e1854db3906e6ba53e267496/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.839307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee21db55ee449d7be269c7eaa58b878, user_id=0a7cd440e1854db3906e6ba53e267496, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:02.845076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:02.865512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1131/2254] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:56:02 2022] PUT /identity/v3/projects/3ee21db55ee449d7be269c7eaa58b878/users/0a7cd440e1854db3906e6ba53e267496/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:56:02.872852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:02.888402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753ad55-0869-4067-9227-81661360e0e4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:02.957611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b695a3a-fe35-426d-9fc6-c02971335bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:02.958301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695a3a-fe35-426d-9fc6-c02971335bae admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.958584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695a3a-fe35-426d-9fc6-c02971335bae admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.958861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b695a3a-fe35-426d-9fc6-c02971335bae admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.959636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695a3a-fe35-426d-9fc6-c02971335bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:02.961297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b695a3a-fe35-426d-9fc6-c02971335bae admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:02.972189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1124/2255] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:56:02.976265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:02.987139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6e318-04c0-4c95-9197-f9c1e0363d28 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:02.992982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:02.993836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:02.994116 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:02.994392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] PATH_INFO: `/v3/projects/3ee21db55ee449d7be269c7eaa58b878/users/0a7cd440e1854db3906e6ba53e267496/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:02.996787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee21db55ee449d7be269c7eaa58b878, user_id=0a7cd440e1854db3906e6ba53e267496, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:02.999522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:03.011411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1132/2256] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:56:02 2022] PUT /identity/v3/projects/3ee21db55ee449d7be269c7eaa58b878/users/0a7cd440e1854db3906e6ba53e267496/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:56:10.483923 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dbb7f-33fa-4ecc-a9e9-0a6b1d6bc1a0 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:10.484482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dbb7f-33fa-4ecc-a9e9-0a6b1d6bc1a0 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:10.484775 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dbb7f-33fa-4ecc-a9e9-0a6b1d6bc1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:10.531326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-241dbb7f-33fa-4ecc-a9e9-0a6b1d6bc1a0 None None] MFA Rules not processed for user `0a7cd440e1854db3906e6ba53e267496`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:10.532475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-241dbb7f-33fa-4ecc-a9e9-0a6b1d6bc1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:10.579324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:10.659019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1125/2257] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:56:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:56:10.682003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:10.684290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:10.763621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71099f25-4e6d-4c1a-8997-9a23c9729109 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:10.768160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:10.768902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:10.769184 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:10.769459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:10.861548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:10.863026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:10.877275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1133/2258] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:56:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:56:11.100204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda50e3-ee4a-42ca-a01d-7bd639b3a59f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:11.100603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda50e3-ee4a-42ca-a01d-7bd639b3a59f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:11.100965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda50e3-ee4a-42ca-a01d-7bd639b3a59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:11.121238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2fda50e3-ee4a-42ca-a01d-7bd639b3a59f None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:11.122507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2fda50e3-ee4a-42ca-a01d-7bd639b3a59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:11.184870 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1126/2259] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:56:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:56:11.199549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:11.201157 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:11.261343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:11.305606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d34a-ca33-46eb-b777-9113b073dfc4 service glance] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:11.313032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:11.313811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:11.314168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:11.315105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:11.326477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:11.328808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:11.339327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1134/2260] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:56:20.338897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ff99-f519-4502-85d5-13b75d69cf09 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:20.339543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ff99-f519-4502-85d5-13b75d69cf09 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:20.339735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662ff99-f519-4502-85d5-13b75d69cf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:20.389255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3662ff99-f519-4502-85d5-13b75d69cf09 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:20.391226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3662ff99-f519-4502-85d5-13b75d69cf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:20.404614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1127/2261] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:56:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:20.424238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:20.449940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80775b3f-d569-4b73-a662-df5ba30c053c service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:20.460685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67de6c-d506-4a7f-8980-2f6b8ab60b76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:20.461544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67de6c-d506-4a7f-8980-2f6b8ab60b76 service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:20.461824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67de6c-d506-4a7f-8980-2f6b8ab60b76 service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:20.462229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67de6c-d506-4a7f-8980-2f6b8ab60b76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:20.494396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67de6c-d506-4a7f-8980-2f6b8ab60b76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:20.499412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67de6c-d506-4a7f-8980-2f6b8ab60b76 service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:20.523862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1135/2262] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:56:22.871366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:22.872513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:22.873133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:22.914508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] MFA Rules not processed for user `030fcf8f4e654d08ac9c7e5dcb3eddd9`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:22.916031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:22.930751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1128/2263] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:56:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:27.899795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd4430-c00e-4259-8784-713ccb6dcdbf None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:27.901496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd4430-c00e-4259-8784-713ccb6dcdbf None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:27.902031 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd4430-c00e-4259-8784-713ccb6dcdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:27.925399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-34fd4430-c00e-4259-8784-713ccb6dcdbf None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:27.927028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-34fd4430-c00e-4259-8784-713ccb6dcdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:27.936883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1136/2264] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:56:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:56:27.957247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:27.973463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d25cab-2608-4906-b01c-2e6e6a143f3b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:27.982118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793b714-7168-47a9-a522-797295f8824f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:27.982937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793b714-7168-47a9-a522-797295f8824f service swift] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:27.983355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793b714-7168-47a9-a522-797295f8824f service swift] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:27.983739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793b714-7168-47a9-a522-797295f8824f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:27.996403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793b714-7168-47a9-a522-797295f8824f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:27.998179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793b714-7168-47a9-a522-797295f8824f service swift] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:28.014111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1129/2265] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:56:37.918343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=21 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:38.009537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=50 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:46.239944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:46.240670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:46.240670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:46.260165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] MFA Rules not processed for user `c58389cb50e140d69f1d4c99d6d0e182`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:46.261449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:46.329506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1137/2266] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:56:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:46.460829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd839977-b929-4de0-ad7f-002db7c19a2e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:46.461463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd839977-b929-4de0-ad7f-002db7c19a2e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:46.462041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd839977-b929-4de0-ad7f-002db7c19a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:46.488616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-cd839977-b929-4de0-ad7f-002db7c19a2e None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:46.490007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-cd839977-b929-4de0-ad7f-002db7c19a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:46.499045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1130/2267] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:56:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:46.513143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:46.527741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7871dd-2d27-4b9b-8f52-ec62442649c9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:46.535025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:46.536302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:46.536437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:46.537205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:46.547827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:46.549418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:46.558346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1138/2268] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:56:47.134554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5d55-4d35-48f4-87c8-7e2ddf8cccd7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:47.135096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5d55-4d35-48f4-87c8-7e2ddf8cccd7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:47.135483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed5d55-4d35-48f4-87c8-7e2ddf8cccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:47.155568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c7ed5d55-4d35-48f4-87c8-7e2ddf8cccd7 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:47.157092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c7ed5d55-4d35-48f4-87c8-7e2ddf8cccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:47.165520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1131/2269] 10.222.0.86 () {58 vars in 967 bytes} [Wed Jul 20 21:56:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:47.179681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:47.190370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db11694-2791-4b13-a64a-2e09d030c0b4 service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:47.196045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:47.196695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:47.196979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:47.197257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:47.205981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:47.207656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:47.216312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1139/2270] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:56:47.450268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba53345-073f-42bb-9c93-5afed4cdf096 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:47.450641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba53345-073f-42bb-9c93-5afed4cdf096 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:47.450985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba53345-073f-42bb-9c93-5afed4cdf096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:47.470055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-eba53345-073f-42bb-9c93-5afed4cdf096 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:47.471304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-eba53345-073f-42bb-9c93-5afed4cdf096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:47.478897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1132/2271] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:56:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:47.490670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:47.501519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3f75d-7762-45cf-acab-5c345824254d service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:47.507598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:47.508244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:47.508523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:47.508960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:47.517919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:47.519726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:47.529212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1140/2272] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:56:47.906510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:47.906874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:47.907344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:47.927317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:47.929227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:47.937060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1133/2273] 10.222.0.86 () {58 vars in 967 bytes} [Wed Jul 20 21:56:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:47.951681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:47.965479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773f8e9-adbb-46b6-bb1a-d50cb02bb2b2 service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:47.970654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc313a49-2a51-4575-ad9d-56c0fd90733f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:47.971332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc313a49-2a51-4575-ad9d-56c0fd90733f service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:47.971618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc313a49-2a51-4575-ad9d-56c0fd90733f service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:47.971895 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc313a49-2a51-4575-ad9d-56c0fd90733f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:47.981523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc313a49-2a51-4575-ad9d-56c0fd90733f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:47.983143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc313a49-2a51-4575-ad9d-56c0fd90733f service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:47.992781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1141/2274] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 21:56:50.109900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:50.175626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f49b3b-64ae-4560-9062-fd79681bb387 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:50.180601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:50.181460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:50.181745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:50.182023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] PATH_INFO: `/v3/users/c9b96b7a3a2142528dd94d5eaf8de9aa` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:50.183245 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b96b7a3a2142528dd94d5eaf8de9aa)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:50.185158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:50.232113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b96b7a3a2142528dd94d5eaf8de9aa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.244198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b96b7a3a2142528dd94d5eaf8de9aa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.251285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b96b7a3a2142528dd94d5eaf8de9aa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.259935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b96b7a3a2142528dd94d5eaf8de9aa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.260553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b96b7a3a2142528dd94d5eaf8de9aa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.266428 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1134/2275] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:56:50 2022] DELETE /identity/v3/users/c9b96b7a3a2142528dd94d5eaf8de9aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:56:50.270937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:50.271477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:50.271573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:50.291211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:50.292429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:50.355812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1142/2276] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:50.366908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:50.368387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:50.429023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec096cf-6bd2-4598-851a-95f1c58dca0d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:50.432778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2818ca4-bea1-40d8-91fb-96349a3bed6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:50.433426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2818ca4-bea1-40d8-91fb-96349a3bed6f service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:50.433710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2818ca4-bea1-40d8-91fb-96349a3bed6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:50.433984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2818ca4-bea1-40d8-91fb-96349a3bed6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:50.441737 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:50.445298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2818ca4-bea1-40d8-91fb-96349a3bed6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:50.447569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2818ca4-bea1-40d8-91fb-96349a3bed6f service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:50.458921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1135/2277] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:56:50.655516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:50.657315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:50.686876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a432016-c389-4cd1-a3c1-45efe83e802e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:50.694589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:50.696004 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:50.696210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:50.696416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] PATH_INFO: `/v3/projects/b4eb21113d83407db43d77aafcb5a33c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:50.699712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4eb21113d83407db43d77aafcb5a33c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:50.701660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:50.784840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:56:50.786311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4eb21113d83407db43d77aafcb5a33c', 'request_id': 'req-304bb130-f404-4628-9dfe-40033fa1ec86'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.797024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4eb21113d83407db43d77aafcb5a33c', 'request_id': 'req-304bb130-f404-4628-9dfe-40033fa1ec86'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.805310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4eb21113d83407db43d77aafcb5a33c', 'request_id': 'req-304bb130-f404-4628-9dfe-40033fa1ec86'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.821095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] The token cache is being invalidate because project b4eb21113d83407db43d77aafcb5a33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:56:50.821409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.822043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-304bb130-f404-4628-9dfe-40033fa1ec86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:56:50.824055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1143/2278] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:56:50 2022] DELETE /identity/v3/projects/b4eb21113d83407db43d77aafcb5a33c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:56:51.133520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f85b5-2dc5-4ef5-b7c3-5a42cdbce4a8 None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.134247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f85b5-2dc5-4ef5-b7c3-5a42cdbce4a8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.134455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f85b5-2dc5-4ef5-b7c3-5a42cdbce4a8 None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.135901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1136/2279] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:56:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:56:51.141938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4de50e-9932-4380-a349-09c9579471a4 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.142303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4de50e-9932-4380-a349-09c9579471a4 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.142641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4de50e-9932-4380-a349-09c9579471a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.162517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5d4de50e-9932-4380-a349-09c9579471a4 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:51.164138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5d4de50e-9932-4380-a349-09c9579471a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:51.243020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1144/2280] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:56:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:51.254864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f85b5-2dc5-4ef5-b7c3-5a42cdbce4a8 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:51.257709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f82f85b5-2dc5-4ef5-b7c3-5a42cdbce4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:51.323852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f85b5-2dc5-4ef5-b7c3-5a42cdbce4a8 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:51.327914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d2fad-3048-4790-83fb-a13cbf3fed05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:51.328501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d2fad-3048-4790-83fb-a13cbf3fed05 service glance] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.328836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d2fad-3048-4790-83fb-a13cbf3fed05 service glance] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.329051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d2fad-3048-4790-83fb-a13cbf3fed05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.340310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d2fad-3048-4790-83fb-a13cbf3fed05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:51.342096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d2fad-3048-4790-83fb-a13cbf3fed05 service glance] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:51.353503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1137/2281] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:56:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:56:51.422924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbb332-c25d-495f-80ca-9295bf8c030e None None] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.423598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbb332-c25d-495f-80ca-9295bf8c030e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.423598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dbb332-c25d-495f-80ca-9295bf8c030e None None] PATH_INFO: `/` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.424917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1145/2282] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:56:51.430432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d26e0-723f-4d8d-abb2-6e7f18c06531 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.430716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d26e0-723f-4d8d-abb2-6e7f18c06531 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.430993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d26e0-723f-4d8d-abb2-6e7f18c06531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.450037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-5e5d26e0-723f-4d8d-abb2-6e7f18c06531 None None] MFA Rules not processed for user `7f38c9f8b5ea438dbae61a15c1e84e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:51.451315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5e5d26e0-723f-4d8d-abb2-6e7f18c06531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:51.503676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1138/2283] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:56:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:51.517627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbb332-c25d-495f-80ca-9295bf8c030e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:51.519357 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a6dbb332-c25d-495f-80ca-9295bf8c030e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:51.532975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dbb332-c25d-495f-80ca-9295bf8c030e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:51.536574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8640f0-4955-47cd-95b9-f8e384e25150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:51.537242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8640f0-4955-47cd-95b9-f8e384e25150 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.538059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8640f0-4955-47cd-95b9-f8e384e25150 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.538368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8640f0-4955-47cd-95b9-f8e384e25150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.545629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8640f0-4955-47cd-95b9-f8e384e25150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:51.547084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8640f0-4955-47cd-95b9-f8e384e25150 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:51.555981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1146/2284] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:56:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:56:51.659869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380825-cbba-4be6-8cca-906b34fe016a None None] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.660236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380825-cbba-4be6-8cca-906b34fe016a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.660588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2380825-cbba-4be6-8cca-906b34fe016a None None] PATH_INFO: `/` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.662725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1139/2285] 10.222.0.86 () {54 vars in 921 bytes} [Wed Jul 20 21:56:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 21:56:51.672120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e3b24-6b84-4ce8-a36a-e0b20b7ea4d7 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.673737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e3b24-6b84-4ce8-a36a-e0b20b7ea4d7 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.673737 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e3b24-6b84-4ce8-a36a-e0b20b7ea4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.698494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7f2e3b24-6b84-4ce8-a36a-e0b20b7ea4d7 None None] MFA Rules not processed for user `e06439f92e494fb4a68ffee7bdb0ffc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:51.711579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7f2e3b24-6b84-4ce8-a36a-e0b20b7ea4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:51.747826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1147/2286] 10.222.0.86 () {58 vars in 1018 bytes} [Wed Jul 20 21:56:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:51.753890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2380825-cbba-4be6-8cca-906b34fe016a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:51.766705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2380825-cbba-4be6-8cca-906b34fe016a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:51.774962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:51.775723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:51.776123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:51.776555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:51.789197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:51.790640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:51.803770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1140/2287] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:56:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:56:55.724444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b58461-63de-463e-a14c-7d74185e0b79 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:55.725574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b58461-63de-463e-a14c-7d74185e0b79 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:55.726248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b58461-63de-463e-a14c-7d74185e0b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:55.746855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c7b58461-63de-463e-a14c-7d74185e0b79 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:56:55.748464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c7b58461-63de-463e-a14c-7d74185e0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:55.814048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1148/2288] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 21:56:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:56:55.841790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:56:55.854212 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:56:55.937955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e8d12-2e63-4e31-80d7-eefdc467e0de service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:56:55.945227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:56:55.945896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:56:55.946186 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:56:55.946551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:56:55.955429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:56:55.957025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:56:55.966181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1141/2289] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:56:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:57:05.807035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=72,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:57:05.962851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:57:27.775119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:57:27.776015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:57:27.776191 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:57:27.794198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:57:27.795502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:57:27.802756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1149/2290] 10.222.0.86 () {58 vars in 967 bytes} [Wed Jul 20 21:57:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:57:27.828418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:57:27.839048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2c321-7f75-4540-8df9-7085e19abb06 service swift] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:57:27.844283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:57:27.844924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:57:27.845206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:57:27.845484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:57:27.854096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:57:27.855709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:57:27.863742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1142/2291] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 21:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 21:57:34.204091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:57:34.205943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:57:34.218928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cb256-cc5a-46af-89c0-5ce9a0d70b0e None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:57:34.222579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:57:34.223274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:57:34.223563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:57:34.223840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:57:34.226011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:57:34.297682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:57:34.299181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:57:34.310104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1150/2292] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:57:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:57:37.860572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:57:39.843938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:57:39.853851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb238d4-e668-4317-85e9-258ea611b38f service swift] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:57:39.857822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:57:39.858473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:57:39.858750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:57:39.859024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:57:39.860875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:57:39.875892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:57:39.877592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae9ab4-6c69-4b71-8acf-43bbda91beda service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:57:39.890802 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1143/2293] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:57:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:57:42.247821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:57:42.256993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5ebb6a-d559-4f4d-923d-fd9806f1a3d5 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:57:42.260660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:57:42.261297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:57:42.261581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:57:42.261859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:57:42.263659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:57:42.277988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:57:42.279473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:57:42.290956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1151/2294] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:57:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:57:49.885477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:57:52.285631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=39 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:57:57.931401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:57:57.931920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:57:57.932027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:57:57.949629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] MFA Rules not processed for user `1a0d4ce0ac4d47c8a184ea0291461379`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:57:57.950759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:57:58.005466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1144/2295] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:57:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:58:00.688081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:00.697394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6317eb30-172d-4a52-a29d-bee85ee6c237 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:00.701257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:00.701897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:00.702293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:00.702495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:00.704491 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:00.716501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:00.717930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:00.732949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1152/2296] 10.222.0.86 () {60 vars in 1396 bytes} [Wed Jul 20 21:58:00 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:02.811869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:02.813370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:02.829704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069e663-4f76-4a03-82f5-bc78f33ee7df None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:02.835816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:02.836448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:02.836728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:02.837002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:02.838716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:02.901814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:02.903464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:02.913195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1145/2297] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:10.722840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:58:12.908349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=53 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:58:31.477414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:31.486602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b50088-c943-4d02-a970-68808a89a532 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:31.490429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:31.491079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:31.491373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:31.491648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:31.493797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:31.503718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:31.505814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:31.515465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1153/2298] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:31.802827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:31.804363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:31.818932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e8fb9-54f7-49d0-9726-6209cc701758 service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:31.825392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:31.826043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:31.826327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:31.826603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:31.828571 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:31.892943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:31.894702 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:31.905682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1146/2299] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:33.293490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:33.307483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7061dff-1c9e-405d-ac2f-450c33a21365 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:33.312878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:33.313533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:33.313809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:33.314082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:33.315857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:33.391074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:33.392725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c370e23-b585-4066-b200-de251e494c99 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:33.402834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1154/2300] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:36.961598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:36.977000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0ddee-5b30-4de0-9637-c0144e6f2dfa service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:36.983690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:36.984424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:36.984657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:36.985163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:36.987194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:37.002943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:37.005154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:37.026227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1147/2301] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:43.397789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=34 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:58:45.428354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:45.428992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:45.429103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:45.447812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] MFA Rules not processed for user `fb27ee14314f42cc87e8e6bac2031c70`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:58:45.449770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:45.457180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1155/2302] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:58:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:58:46.170717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:46.183705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9af4-c6fe-4e2c-8af8-876f2b91ad33 service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:46.187464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:46.188207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:46.189916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:46.189916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:46.190821 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:46.201270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:46.202800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1359-f651-48c0-a51a-ec914c4ebde1 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:46.211252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1148/2303] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:47.316590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:47.318141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:47.408301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc622103-c939-4a5f-b751-ec6293cdbe23 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:47.413404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:47.414064 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:47.414345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:47.414619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:47.417231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:47.429196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:47.430809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:47.441814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1156/2304] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:52.922262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:52.922773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:52.922882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:52.928783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=41 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:58:52.941208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] MFA Rules not processed for user `c58389cb50e140d69f1d4c99d6d0e182`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:58:52.942433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:52.949966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1149/2305] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:58:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:58:53.262628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:53.272193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e070e7-2634-40e3-92a6-fd018de4ab77 service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:53.275927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:53.276570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:53.276852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:53.277131 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:53.278914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:53.288492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:53.289913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:53.297981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1157/2306] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:58:59.164450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:58:59.178786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11678ef4-4dae-487d-b9fe-0372c7947ae1 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:58:59.185385 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:58:59.186040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:58:59.186317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:58:59.186589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:58:59.188496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:58:59.201887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:58:59.203510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcadb2d1-6410-4b3c-ae5a-05fc65c0c163 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:58:59.214092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1150/2307] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:58:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:03.294416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=41 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:03.459691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:03.475937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ba497-34fa-4389-81e7-7c480574514c service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:03.483755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:03.484641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:03.484954 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:03.487104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:03.487725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:03.505461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:03.507611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:03.524007 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1158/2308] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:09.209586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:13.519925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:19.940800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:19.941321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:19.941426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:19.959752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] MFA Rules not processed for user `60f7f43c067e4b819aeb488e9f09197a`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:19.960961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:19.968477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1151/2309] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:29.954774 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:33.419846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:33.430709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828435e9-5c56-4c6f-947d-5e8e8eb175d1 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:33.434300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:33.435216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:33.435494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:33.435767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:33.437707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:33.512605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:33.514063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:33.526718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1159/2310] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:37.271266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:37.300149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9496eaa-8485-4f65-b5a6-af0896970654 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:37.306783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:37.307577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:37.307841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:37.308303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:37.310785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:37.332175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:37.335369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:37.339946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:37.341480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:37.354559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1152/2311] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:37.359655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe9283-3231-47a5-a8a5-82eb6f4056f7 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:37.377321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:37.378221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:37.378506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:37.378784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] PATH_INFO: `/v3/users/1a0d4ce0ac4d47c8a184ea0291461379` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:37.380086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0d4ce0ac4d47c8a184ea0291461379)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:37.382537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:37.431092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0d4ce0ac4d47c8a184ea0291461379'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.443515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0d4ce0ac4d47c8a184ea0291461379'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.451447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0d4ce0ac4d47c8a184ea0291461379'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.452213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0d4ce0ac4d47c8a184ea0291461379'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.452887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec753247-570a-4592-9960-3c4f58d89467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0d4ce0ac4d47c8a184ea0291461379'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.467989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1160/2312] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:37 2022] DELETE /identity/v3/users/1a0d4ce0ac4d47c8a184ea0291461379 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:37.657395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:37.671423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f5d6e-ad32-4947-b8b2-76e0c6fc0223 service placement] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:37.740582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:37.741455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:37.741739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:37.742016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] PATH_INFO: `/v3/projects/80320a3595ce4d4cb7c468b0fbec6580` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:37.743409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80320a3595ce4d4cb7c468b0fbec6580)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:37.745319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:37.801893 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80320a3595ce4d4cb7c468b0fbec6580', 'request_id': 'req-847e1412-1751-4531-ac93-e94ef8c51f25'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.811858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80320a3595ce4d4cb7c468b0fbec6580', 'request_id': 'req-847e1412-1751-4531-ac93-e94ef8c51f25'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.817936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80320a3595ce4d4cb7c468b0fbec6580', 'request_id': 'req-847e1412-1751-4531-ac93-e94ef8c51f25'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.834620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] The token cache is being invalidate because project 80320a3595ce4d4cb7c468b0fbec6580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:37.835023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.835833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-847e1412-1751-4531-ac93-e94ef8c51f25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:37.838319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1153/2313] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:37 2022] DELETE /identity/v3/projects/80320a3595ce4d4cb7c468b0fbec6580 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:37.843717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:37.844073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:37.844407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:37.864133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:37.865526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:37.932403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1161/2314] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:37.941583 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:37.941897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:37.942178 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:37.964345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:37.965587 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:37.973709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1154/2315] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:37.978827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:37.989405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c117b-5ab3-4403-9447-ad7b5fdac29a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:37.994281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:37.994946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:37.995266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:37.995546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:37.996318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:59:37.996657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:37.998267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.000194 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:38.013350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1162/2316] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 21:59:38.018244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.028941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64f6d4-2e31-4e58-aeb5-5a9e82472d6e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.033639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.034298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.034576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.034851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.035573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.037458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.065240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1155/2317] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:38 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:38.069330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.080545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb158af-b075-4087-ba27-d5185b56014d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.142106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.142689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.142973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.143369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.144082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.146000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.166847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1163/2318] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:38 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:38.171448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.182741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de695f0-361a-4fe6-be36-0add0a09fcc7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.187525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.188197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.188483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.188811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.189528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.191208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.202615 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1156/2319] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:59:38.206747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.217522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d155067-8220-483f-887d-b12e4a129e16 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.222209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8b036-da41-479f-8964-d89e460e97c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.222864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8b036-da41-479f-8964-d89e460e97c3 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.223174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8b036-da41-479f-8964-d89e460e97c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.223449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8b036-da41-479f-8964-d89e460e97c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.224215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8b036-da41-479f-8964-d89e460e97c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.225823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8b036-da41-479f-8964-d89e460e97c3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.236700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1164/2320] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:38.241259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.252029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27340f-df7d-47e5-812e-a99313137032 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.256793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.257663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.258513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.258966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] PATH_INFO: `/v3/projects/b8143a1bbc54412b87eafdf7c9956115/users/202d19dcd72740a5b97095fcde8629fe/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.274706 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8143a1bbc54412b87eafdf7c9956115, user_id=202d19dcd72740a5b97095fcde8629fe, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.277397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.290538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1157/2321] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:38 2022] PUT /identity/v3/projects/b8143a1bbc54412b87eafdf7c9956115/users/202d19dcd72740a5b97095fcde8629fe/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:38.300991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.301282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.301556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.328848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] MFA Rules not processed for user `202d19dcd72740a5b97095fcde8629fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:38.330095 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:38.396608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1165/2322] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:38.401647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.415513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:38.469168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4cfa-283e-4226-ae9e-0915c366ebdc admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.474210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.474877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.475197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.475482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.476191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.478243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.509264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1158/2323] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:38 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:38.511200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.522432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70b75e-da56-47e5-b747-473a7bac0df5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.583227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.583902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.584182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.584456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.585154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.587642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.595463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:38.614750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1166/2324] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:38 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:38.615978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.628972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78367d0e-e55a-4fae-8880-aea949f9e1ee admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.634221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.634899 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.635221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.635502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.636287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.637895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.649997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1159/2325] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 21:59:38.654159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.666039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a00eea-d4c2-44da-98c1-b9ae331672f8 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.670924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.671868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.672149 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.672427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] PATH_INFO: `/v3/projects/cdd9f86293e84ee38eed86f7c55e55ea/users/fe0f3d6e4f864d90a5f6ffe4b0efeefa/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.687882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd9f86293e84ee38eed86f7c55e55ea, user_id=fe0f3d6e4f864d90a5f6ffe4b0efeefa, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.690564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.704310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1167/2326] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:38 2022] PUT /identity/v3/projects/cdd9f86293e84ee38eed86f7c55e55ea/users/fe0f3d6e4f864d90a5f6ffe4b0efeefa/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:38.708438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.720147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8fcd2-5e59-44f4-9864-fa54506e7684 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.781464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aa3f7-6672-48a8-a831-a452ee517030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.782129 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aa3f7-6672-48a8-a831-a452ee517030 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.782414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aa3f7-6672-48a8-a831-a452ee517030 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.782679 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aa3f7-6672-48a8-a831-a452ee517030 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.783589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aa3f7-6672-48a8-a831-a452ee517030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.785177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aa3f7-6672-48a8-a831-a452ee517030 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.796270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1160/2327] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:38.800530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.811146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca8d18-88e1-4927-9ecb-4003e83b7af2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:38.815859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:38.816741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.817022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.817295 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] PATH_INFO: `/v3/projects/cdd9f86293e84ee38eed86f7c55e55ea/users/fe0f3d6e4f864d90a5f6ffe4b0efeefa/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.819684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdd9f86293e84ee38eed86f7c55e55ea, user_id=fe0f3d6e4f864d90a5f6ffe4b0efeefa, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:38.823039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:38.833730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1168/2328] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:38 2022] PUT /identity/v3/projects/cdd9f86293e84ee38eed86f7c55e55ea/users/fe0f3d6e4f864d90a5f6ffe4b0efeefa/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:38.844761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:38.845075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:38.845361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:38.872497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] MFA Rules not processed for user `fe0f3d6e4f864d90a5f6ffe4b0efeefa`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:38.873721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:38.951381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1161/2329] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:38.964258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:38.965988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:39.024483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85cb7e-d01b-43fe-a2ad-386c89b01616 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:39.028226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3095a1-811d-4966-aed7-6fe5dfd8c768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:39.028866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3095a1-811d-4966-aed7-6fe5dfd8c768 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:39.029142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3095a1-811d-4966-aed7-6fe5dfd8c768 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:39.029411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3095a1-811d-4966-aed7-6fe5dfd8c768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:39.039252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3095a1-811d-4966-aed7-6fe5dfd8c768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:39.040745 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3095a1-811d-4966-aed7-6fe5dfd8c768 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:39.050601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1169/2330] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:39.492961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:39.503596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:39.563621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077cd67-bff9-4a19-9f6a-3d2a5bf86b09 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:39.568329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:39.569206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:39.569498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:39.569759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] PATH_INFO: `/v3/users/202d19dcd72740a5b97095fcde8629fe` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:39.572006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] RBAC: Authorizing `identity:delete_user(user_id=202d19dcd72740a5b97095fcde8629fe)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:39.574567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:39.634600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '202d19dcd72740a5b97095fcde8629fe'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:39.650830 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '202d19dcd72740a5b97095fcde8629fe'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:39.657998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '202d19dcd72740a5b97095fcde8629fe'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:39.666254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '202d19dcd72740a5b97095fcde8629fe'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:39.667287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '202d19dcd72740a5b97095fcde8629fe'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:39.673614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1162/2331] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:39 2022] DELETE /identity/v3/users/202d19dcd72740a5b97095fcde8629fe => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:39.678755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:39.679296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:39.679509 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:39.698514 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:39.699773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:39.764122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1170/2332] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:39.773793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:39.775262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:39.834197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f54731-04b5-42e0-aa0a-e4dec94622da admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:39.837968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:39.838621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:39.838914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:39.839232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:39.848782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:39.850223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:39.861048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1163/2333] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:40.149375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:40.150981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:40.163449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5174b-819d-474c-957a-7f21400d3fb9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:40.170796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:40.172582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:40.173070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:40.173556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] PATH_INFO: `/v3/projects/b8143a1bbc54412b87eafdf7c9956115` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:40.175732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8143a1bbc54412b87eafdf7c9956115)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:40.179787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:40.236567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8143a1bbc54412b87eafdf7c9956115', 'request_id': 'req-3a968476-7de3-4d69-b855-ed93fff81a20'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.238592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:40.243826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8143a1bbc54412b87eafdf7c9956115', 'request_id': 'req-3a968476-7de3-4d69-b855-ed93fff81a20'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.251243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8143a1bbc54412b87eafdf7c9956115', 'request_id': 'req-3a968476-7de3-4d69-b855-ed93fff81a20'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.263998 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] The token cache is being invalidate because project b8143a1bbc54412b87eafdf7c9956115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:40.264279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.264897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.266449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1171/2334] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:40 2022] DELETE /identity/v3/projects/b8143a1bbc54412b87eafdf7c9956115 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:40.270804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:40.281200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991794a-37c5-4f46-9543-477055015ebf service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:40.310711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:40.344388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:40.345263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:40.345540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:40.345807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] PATH_INFO: `/v3/users/fe0f3d6e4f864d90a5f6ffe4b0efeefa` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:40.347007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0f3d6e4f864d90a5f6ffe4b0efeefa)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:40.348924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:40.396968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0f3d6e4f864d90a5f6ffe4b0efeefa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.410364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0f3d6e4f864d90a5f6ffe4b0efeefa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.417477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0f3d6e4f864d90a5f6ffe4b0efeefa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.425170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0f3d6e4f864d90a5f6ffe4b0efeefa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.425995 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e4e30b52-221e-40a3-b107-9c7f100049d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0f3d6e4f864d90a5f6ffe4b0efeefa'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.432515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1164/2335] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:40 2022] DELETE /identity/v3/users/fe0f3d6e4f864d90a5f6ffe4b0efeefa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:40.723476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:40.725022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:40.741606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a968476-7de3-4d69-b855-ed93fff81a20 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:40.801428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:40.802267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:40.802557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:40.802836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] PATH_INFO: `/v3/projects/cdd9f86293e84ee38eed86f7c55e55ea` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:40.804101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdd9f86293e84ee38eed86f7c55e55ea)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:40.806630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:40.853771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdd9f86293e84ee38eed86f7c55e55ea', 'request_id': 'req-c5471e19-3043-40f4-a658-73d664c383ec'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.860055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdd9f86293e84ee38eed86f7c55e55ea', 'request_id': 'req-c5471e19-3043-40f4-a658-73d664c383ec'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.865559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdd9f86293e84ee38eed86f7c55e55ea', 'request_id': 'req-c5471e19-3043-40f4-a658-73d664c383ec'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.878808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] The token cache is being invalidate because project cdd9f86293e84ee38eed86f7c55e55ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:40.879232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.879864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c5471e19-3043-40f4-a658-73d664c383ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:40.881410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1172/2336] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:40 2022] DELETE /identity/v3/projects/cdd9f86293e84ee38eed86f7c55e55ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:40.888934 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:40.889738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:40.890012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:40.908339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:40.909568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:40.969683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1165/2337] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:40.979535 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:40.979952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:40.980323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.000443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:41.001659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:41.010439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1173/2338] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:41.014944 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.024925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0552af3-dfdd-40d1-8a41-914a38fe67f5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.029647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.030310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.030595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.030905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.031710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:59:41.032022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.033628 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.045763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1166/2339] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:59:41.050423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.060881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4782850-9d6b-468f-9900-1ccf844150f3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.069603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.071236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.071655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.072173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.072983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.075503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.108354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1174/2340] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:41 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:59:41.109935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.121481 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97415c5b-1d9c-4754-bf6f-68679571d6c0 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.155929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:41.181921 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.182604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.182889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.183226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.183922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.185904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.206500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1167/2341] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:41 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:41.210848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.221866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faedf5c-4e8f-41d3-b8cf-d64b48a01433 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.227490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.228456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.228883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.229183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.229950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.231611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.243217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1175/2342] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:41.247749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.258257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626781e-2349-4791-88b8-b983017b7d67 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.263339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759e24d-2894-4fae-894d-1c2ff4890b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.264011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e24d-2894-4fae-894d-1c2ff4890b24 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.264296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e24d-2894-4fae-894d-1c2ff4890b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.264575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759e24d-2894-4fae-894d-1c2ff4890b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.265311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759e24d-2894-4fae-894d-1c2ff4890b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.266915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759e24d-2894-4fae-894d-1c2ff4890b24 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.277541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1168/2343] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:41.281767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.292886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72139aac-37cd-4238-b226-fade720c235c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.297512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.298395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.298677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.298951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] PATH_INFO: `/v3/projects/d5a5b01e7b9f497a8d3778a9fc84d6d9/users/f61e6ba6f2f9406588ab684a434575f3/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.315152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a5b01e7b9f497a8d3778a9fc84d6d9, user_id=f61e6ba6f2f9406588ab684a434575f3, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.317770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.332346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1176/2344] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:41 2022] PUT /identity/v3/projects/d5a5b01e7b9f497a8d3778a9fc84d6d9/users/f61e6ba6f2f9406588ab684a434575f3/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:41.343126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.343416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.343690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.372205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] MFA Rules not processed for user `f61e6ba6f2f9406588ab684a434575f3`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:41.373443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:41.438371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1169/2345] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:41.442869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.452092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:41.508382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e4b87-6270-4b74-af6d-8f3a956658dd admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.513532 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.514203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.514490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.514770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.515507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.517443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.545896 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1177/2346] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:41 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:41.549431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.560108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee3a53-2a23-4cb5-b4f0-decbaf95e384 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.619635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.620213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.620491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.620761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.621458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.623431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.644020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1170/2347] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:41 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:59:41.647957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.660705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb088b56-af0e-4a7e-b9e5-1419fc6d3221 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.667320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.668399 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.668859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.669308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.670547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.673387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.688213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1178/2348] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:41.692640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.702881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525085b9-bef2-4c29-ba08-8e6fe0915148 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.707735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.708624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.708926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.709173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] PATH_INFO: `/v3/projects/e0d6987cbf2644549fddc5605c96bd1f/users/e102f3b4a26e4042b4d4c16575dc1ccc/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.725153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d6987cbf2644549fddc5605c96bd1f, user_id=e102f3b4a26e4042b4d4c16575dc1ccc, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.728188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.741935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1171/2349] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:41 2022] PUT /identity/v3/projects/e0d6987cbf2644549fddc5605c96bd1f/users/e102f3b4a26e4042b4d4c16575dc1ccc/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:41.746242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.757466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd00cf7-caf9-4489-ac37-3bd1ebe6b662 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.816692 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fec2d-2f8c-42db-8a5e-b4655765e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.817344 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fec2d-2f8c-42db-8a5e-b4655765e7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.817629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fec2d-2f8c-42db-8a5e-b4655765e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.817907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fec2d-2f8c-42db-8a5e-b4655765e7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.818641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fec2d-2f8c-42db-8a5e-b4655765e7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.820308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fec2d-2f8c-42db-8a5e-b4655765e7d5 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.830940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1179/2350] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:41.835168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:41.846559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ceb37a-584f-464f-b351-283a2d9e6066 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:41.851208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:41.852124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.852406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.852724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] PATH_INFO: `/v3/projects/e0d6987cbf2644549fddc5605c96bd1f/users/e102f3b4a26e4042b4d4c16575dc1ccc/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.854910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0d6987cbf2644549fddc5605c96bd1f, user_id=e102f3b4a26e4042b4d4c16575dc1ccc, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:41.857574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:41.863347 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:41.870535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1172/2351] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:41 2022] PUT /identity/v3/projects/e0d6987cbf2644549fddc5605c96bd1f/users/e102f3b4a26e4042b4d4c16575dc1ccc/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:41.882234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:41.882540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:41.882808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:41.915830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] MFA Rules not processed for user `e102f3b4a26e4042b4d4c16575dc1ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:41.917053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:41.990379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1180/2352] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:42.002237 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:42.004094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:42.064058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb8969-4f40-453b-8db1-df298bfe713d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:42.067826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067846a-0881-424e-8be8-91231630c5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:42.068482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067846a-0881-424e-8be8-91231630c5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:42.068766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067846a-0881-424e-8be8-91231630c5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:42.069045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067846a-0881-424e-8be8-91231630c5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:42.079225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067846a-0881-424e-8be8-91231630c5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:42.080688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067846a-0881-424e-8be8-91231630c5b5 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:42.094752 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1173/2353] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:42.123158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:42.127670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:42.189739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80479e-4ccc-43fb-96c9-eadbc764d5cf None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:42.194459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:42.195362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:42.195650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:42.195926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] PATH_INFO: `/v3/users/f61e6ba6f2f9406588ab684a434575f3` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:42.197703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61e6ba6f2f9406588ab684a434575f3)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:42.199699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:42.247261 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61e6ba6f2f9406588ab684a434575f3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.260415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61e6ba6f2f9406588ab684a434575f3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.268679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61e6ba6f2f9406588ab684a434575f3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.270300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61e6ba6f2f9406588ab684a434575f3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.270908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61e6ba6f2f9406588ab684a434575f3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.285096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1181/2354] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:42 2022] DELETE /identity/v3/users/f61e6ba6f2f9406588ab684a434575f3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:42.290248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:42.290554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:42.291671 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:42.313087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:42.314333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:42.377520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1174/2355] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:42.388388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:42.389833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:42.448079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8f46a-8141-4395-af5c-d74060754aaf admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:42.451824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:42.452474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:42.452757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:42.453035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:42.462693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:42.464182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:42.474068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1182/2356] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 21:59:42.760517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:42.762338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:42.778126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6efc-830c-4724-bca5-7bbcfda4a2fe None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:42.785940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:42.786962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:42.787389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:42.787746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] PATH_INFO: `/v3/projects/d5a5b01e7b9f497a8d3778a9fc84d6d9` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:42.790422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a5b01e7b9f497a8d3778a9fc84d6d9)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:42.793021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:42.849640 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a5b01e7b9f497a8d3778a9fc84d6d9', 'request_id': 'req-1e55a8a0-7736-433e-a6c2-fa8a6556a924'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.856632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a5b01e7b9f497a8d3778a9fc84d6d9', 'request_id': 'req-1e55a8a0-7736-433e-a6c2-fa8a6556a924'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.862954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a5b01e7b9f497a8d3778a9fc84d6d9', 'request_id': 'req-1e55a8a0-7736-433e-a6c2-fa8a6556a924'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.877889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] The token cache is being invalidate because project d5a5b01e7b9f497a8d3778a9fc84d6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:42.878207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.879036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:42.881083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1175/2357] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:42 2022] DELETE /identity/v3/projects/d5a5b01e7b9f497a8d3778a9fc84d6d9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:42.886476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:42.896410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd515b00-66c7-46cf-bede-3ca669915f6e service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:42.918767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:42.974583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:42.975503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:42.975779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:42.976051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] PATH_INFO: `/v3/users/e102f3b4a26e4042b4d4c16575dc1ccc` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:42.977355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e102f3b4a26e4042b4d4c16575dc1ccc)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:42.980049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.031275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e102f3b4a26e4042b4d4c16575dc1ccc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.047608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e102f3b4a26e4042b4d4c16575dc1ccc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.054530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e102f3b4a26e4042b4d4c16575dc1ccc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.055286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e102f3b4a26e4042b4d4c16575dc1ccc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.055888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-5d1ae08d-c241-43f7-9c80-9218cc1bdb8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e102f3b4a26e4042b4d4c16575dc1ccc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.069723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1183/2358] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:42 2022] DELETE /identity/v3/users/e102f3b4a26e4042b4d4c16575dc1ccc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:43.371576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.373110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:43.387710 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55a8a0-7736-433e-a6c2-fa8a6556a924 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.414140 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:43.457906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.459261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.459762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.460227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] PATH_INFO: `/v3/projects/e0d6987cbf2644549fddc5605c96bd1f` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.462130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0d6987cbf2644549fddc5605c96bd1f)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:43.466044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.517740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0d6987cbf2644549fddc5605c96bd1f', 'request_id': 'req-1db5f9c9-fc64-4201-b893-3ad7925eeff8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.524728 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0d6987cbf2644549fddc5605c96bd1f', 'request_id': 'req-1db5f9c9-fc64-4201-b893-3ad7925eeff8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.531474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0d6987cbf2644549fddc5605c96bd1f', 'request_id': 'req-1db5f9c9-fc64-4201-b893-3ad7925eeff8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.545284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] The token cache is being invalidate because project e0d6987cbf2644549fddc5605c96bd1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:43.545641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.546199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1db5f9c9-fc64-4201-b893-3ad7925eeff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:43.547780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1176/2359] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:43 2022] DELETE /identity/v3/projects/e0d6987cbf2644549fddc5605c96bd1f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:43.552937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.553229 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.553512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.572494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:43.573728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:43.647529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1184/2360] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 21:59:43.656126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.656412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.656687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.688305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:43.689542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:43.698879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1177/2361] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:43.703285 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.714165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5293980-6aba-456f-8d7f-726cc6a1bc93 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.719453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.720118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.720418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.720700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.721437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:59:43.721747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:43.723424 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.735829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1185/2362] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:59:43.740398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.751715 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29d8a7-f605-42b3-a31f-ca5e3209f063 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.756240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.756908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.757193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.757468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.758167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:43.760626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.787867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1178/2363] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:43 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:59:43.793409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.804156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f54248-a6b1-4c52-a099-d6e8291b6498 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.834101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:43.872254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.872931 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.873209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.873487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.874457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:43.876462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.899521 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1186/2364] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:43 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 21:59:43.904416 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.915283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d58ee7-00bf-4127-9ddc-71ad8d7509ba admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.920807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.921499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.921778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.922050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.922786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:43.924573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.935876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1179/2365] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:43.940226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.954015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf2c0d-c653-4383-a81c-29c376fab2e0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.959594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df337f-075f-49e6-8d62-a424c5971c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.960331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df337f-075f-49e6-8d62-a424c5971c63 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.960805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df337f-075f-49e6-8d62-a424c5971c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:43.961099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df337f-075f-49e6-8d62-a424c5971c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:43.961926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df337f-075f-49e6-8d62-a424c5971c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:43.964694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df337f-075f-49e6-8d62-a424c5971c63 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:43.975025 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1187/2366] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:43.981296 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:43.992012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534aa9d-6391-4c4d-908d-9e8ef6ae6bf6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:43.997743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:43.999412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:43.999847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:44.000284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] PATH_INFO: `/v3/projects/a62ba414699344debf7b0cdd753418db/users/ad3149f6a9254534a49bb05b01f917dd/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:44.017633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62ba414699344debf7b0cdd753418db, user_id=ad3149f6a9254534a49bb05b01f917dd, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:44.020350 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:44.036367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1180/2367] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:43 2022] PUT /identity/v3/projects/a62ba414699344debf7b0cdd753418db/users/ad3149f6a9254534a49bb05b01f917dd/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:44.047628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:44.047917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:44.048196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:44.076298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] MFA Rules not processed for user `ad3149f6a9254534a49bb05b01f917dd`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:44.077534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:44.145251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1188/2368] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:44.159266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:44.160978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:44.223546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012625c-305d-4f6f-9804-e73a46a688a9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:44.227339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c711b8-db57-4f6b-ab05-a9b2daa402ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:44.228017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c711b8-db57-4f6b-ab05-a9b2daa402ee service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:44.228816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c711b8-db57-4f6b-ab05-a9b2daa402ee service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:44.229096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c711b8-db57-4f6b-ab05-a9b2daa402ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:44.234551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:44.239394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c711b8-db57-4f6b-ab05-a9b2daa402ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:44.241792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c711b8-db57-4f6b-ab05-a9b2daa402ee service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:44.254341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1181/2369] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:47.378190 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:47.440936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb0a70-9e6a-447f-baf2-e4f1261739a0 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:47.445472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:47.446342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:47.446623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:47.447730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] PATH_INFO: `/v3/users/ad3149f6a9254534a49bb05b01f917dd` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:47.448938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad3149f6a9254534a49bb05b01f917dd)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:47.450885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:47.489212 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:47.496162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad3149f6a9254534a49bb05b01f917dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:47.508767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad3149f6a9254534a49bb05b01f917dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:47.516334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad3149f6a9254534a49bb05b01f917dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:47.517039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad3149f6a9254534a49bb05b01f917dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:47.517651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad3149f6a9254534a49bb05b01f917dd'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:47.530683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1189/2370] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:47 2022] DELETE /identity/v3/users/ad3149f6a9254534a49bb05b01f917dd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:47.535324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:47.535826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:47.536130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:47.553939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:47.555301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:47.618478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1182/2371] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:47.628543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:47.629980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:47.689151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c6e1c-510b-4dea-b854-b14ae3af3ca7 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:47.692870 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec6ede-9270-4cd4-a85f-c5d29ac40dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:47.693518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6ede-9270-4cd4-a85f-c5d29ac40dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:47.693800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6ede-9270-4cd4-a85f-c5d29ac40dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:47.694076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec6ede-9270-4cd4-a85f-c5d29ac40dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:47.703246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec6ede-9270-4cd4-a85f-c5d29ac40dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:47.704688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec6ede-9270-4cd4-a85f-c5d29ac40dd6 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:47.714652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1190/2372] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:47.989681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:47.991218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:48.004209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f02f2-3f28-435a-b0a6-07d76c843e2e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.009307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.010255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.010537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.010812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] PATH_INFO: `/v3/projects/a62ba414699344debf7b0cdd753418db` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.012115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62ba414699344debf7b0cdd753418db)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.014161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.064843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62ba414699344debf7b0cdd753418db', 'request_id': 'req-ccf63a58-7557-442c-a096-ae81b9088f59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:48.072249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62ba414699344debf7b0cdd753418db', 'request_id': 'req-ccf63a58-7557-442c-a096-ae81b9088f59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:48.078280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62ba414699344debf7b0cdd753418db', 'request_id': 'req-ccf63a58-7557-442c-a096-ae81b9088f59'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:48.091983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] The token cache is being invalidate because project a62ba414699344debf7b0cdd753418db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:48.092275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:48.092913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ccf63a58-7557-442c-a096-ae81b9088f59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:48.094476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1183/2373] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:47 2022] DELETE /identity/v3/projects/a62ba414699344debf7b0cdd753418db => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:48.099324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.099621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.099901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.118168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:48.119923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:48.180652 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1191/2374] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:48.190446 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.190828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.191215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.210194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:48.211420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:48.220200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1184/2375] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:48.224495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.234830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca43f7-2640-4af9-adc0-2fd50eab8d65 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.239431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.240090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.240372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.240649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.241380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:59:48.241689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.243359 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.255431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1192/2376] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:59:48.259702 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.268875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a63494-5cac-42e4-a561-911e5bc840b7 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.273447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.274085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.274361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.274629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.276057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.277950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.303621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1185/2377] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:48 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:48.308250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.318367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38a438-ec3c-46c0-a044-3c60fb5a6fed admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.362943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:48.378322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.378920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.379216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.379494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.380189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.382117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.402127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1193/2378] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:48 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:48.406360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.417362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151d142-66e2-4120-91dc-ff5d030b0529 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.421914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.422568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.422849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.423167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.423895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.425504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.435889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1186/2379] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:48.439853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.449961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5302f6-f8d6-461e-a2d1-46f016113758 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.454541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795d45-e84c-4429-8d9c-b16b1d9dd7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.455218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795d45-e84c-4429-8d9c-b16b1d9dd7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.455496 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795d45-e84c-4429-8d9c-b16b1d9dd7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.455768 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795d45-e84c-4429-8d9c-b16b1d9dd7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.456486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02795d45-e84c-4429-8d9c-b16b1d9dd7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.458071 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02795d45-e84c-4429-8d9c-b16b1d9dd7d2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.468296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1194/2380] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:48.472360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.482777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3381-e6c1-48e3-8402-35f283655f86 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.487454 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.488321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.488605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.488879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] PATH_INFO: `/v3/projects/01bde391b2914228b5adc710b3d48e74/users/3783e72420e145149437e241c3503133/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.503568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bde391b2914228b5adc710b3d48e74, user_id=3783e72420e145149437e241c3503133, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.506230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.519121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1187/2381] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:48 2022] PUT /identity/v3/projects/01bde391b2914228b5adc710b3d48e74/users/3783e72420e145149437e241c3503133/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 21:59:48.529887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.530183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.530462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.557670 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] MFA Rules not processed for user `3783e72420e145149437e241c3503133`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:48.558904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:48.622620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1195/2382] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:48.634102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:48.636401 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:48.651328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:48.697779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b995a8ee-cc58-4686-bc35-b71da54af8bb admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:48.701487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db37d13-7f91-4adb-acf5-bd33bc08d53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:48.702127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db37d13-7f91-4adb-acf5-bd33bc08d53d service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:48.702407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db37d13-7f91-4adb-acf5-bd33bc08d53d service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:48.702681 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db37d13-7f91-4adb-acf5-bd33bc08d53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:48.711015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db37d13-7f91-4adb-acf5-bd33bc08d53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:48.713098 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db37d13-7f91-4adb-acf5-bd33bc08d53d service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:48.722871 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1188/2383] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:56.792130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:56.853928 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10c1b6-213a-4a4f-9cff-22b14261652b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:56.858440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:56.859332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:56.859614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:56.859886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] PATH_INFO: `/v3/users/3783e72420e145149437e241c3503133` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:56.861070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3783e72420e145149437e241c3503133)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:56.863006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:56.909135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3783e72420e145149437e241c3503133'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:56.917527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:56.922626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3783e72420e145149437e241c3503133'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:56.930180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3783e72420e145149437e241c3503133'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:56.930849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3783e72420e145149437e241c3503133'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:56.932139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3783e72420e145149437e241c3503133'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:56.944441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1196/2384] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 21:59:56 2022] DELETE /identity/v3/users/3783e72420e145149437e241c3503133 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:56.949057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:56.949527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:56.949641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:56.966361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:56.967591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:57.025972 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1189/2385] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:57.035598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.037006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:57.096487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb119e8-fd2e-45b1-b341-7d44e479570b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.100183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e6577-e36a-44cf-bc7f-92fda52aae0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.100859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e6577-e36a-44cf-bc7f-92fda52aae0b service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.101140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e6577-e36a-44cf-bc7f-92fda52aae0b service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.101414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e6577-e36a-44cf-bc7f-92fda52aae0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.111440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e6577-e36a-44cf-bc7f-92fda52aae0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.112869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e6577-e36a-44cf-bc7f-92fda52aae0b service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.123241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1197/2386] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 21:59:57.402938 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.404459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:57.428397 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cba685-97d5-4871-aa51-3f60ddf50790 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.433514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.434897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.436659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.437142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] PATH_INFO: `/v3/projects/01bde391b2914228b5adc710b3d48e74` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.439318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bde391b2914228b5adc710b3d48e74)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.443430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.504240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bde391b2914228b5adc710b3d48e74', 'request_id': 'req-b03aebf6-3a49-4d5e-bc65-b942016c5abd'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:57.519321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bde391b2914228b5adc710b3d48e74', 'request_id': 'req-b03aebf6-3a49-4d5e-bc65-b942016c5abd'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:57.519321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bde391b2914228b5adc710b3d48e74', 'request_id': 'req-b03aebf6-3a49-4d5e-bc65-b942016c5abd'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:57.532515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] The token cache is being invalidate because project 01bde391b2914228b5adc710b3d48e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 21:59:57.532761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:57.533374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b03aebf6-3a49-4d5e-bc65-b942016c5abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 21:59:57.534998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1190/2387] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 21:59:57 2022] DELETE /identity/v3/projects/01bde391b2914228b5adc710b3d48e74 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:57.540082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.540439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.541258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.564797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:57.566649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:57.636752 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1198/2388] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:57.644662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.644958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.645241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.664437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:57.666260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:57.674854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1191/2389] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:57.679709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.691342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a3d3f-30b4-42be-bba0-bc5f9031119a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.696599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.697439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.697716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.697995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.698736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 21:59:57.699038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.700698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.713383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1199/2390] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 21:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 21:59:57.717788 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.728745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa88faa-fd9e-40a5-821d-10d1da2757d0 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.733612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.734285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.734562 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.734836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.735702 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.737766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.774106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1192/2391] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 21:59:57 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:57.779017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.790209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb106-adc9-4151-b84e-4f0af4cb1d42 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.850924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.851637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.851905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.852182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.853508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.855790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.867572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:57.886968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1200/2392] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 21:59:57 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 21:59:57.888372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.900481 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18185344-b49b-4c2e-9206-aaf5d52fdfde admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.905510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.906211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.906492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.906768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.907555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.909220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.915342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 21:59:57.925138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1193/2393] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:57.926487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.938608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6768727-d44f-4ed0-8227-94a34158145d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.943628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63908f0-286b-4db9-a506-83215a710d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.944320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63908f0-286b-4db9-a506-83215a710d23 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.945252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63908f0-286b-4db9-a506-83215a710d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.945543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63908f0-286b-4db9-a506-83215a710d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:57.946324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63908f0-286b-4db9-a506-83215a710d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:57.948132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63908f0-286b-4db9-a506-83215a710d23 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:57.966298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1201/2394] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 21:59:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 21:59:57.967732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:57.980400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0f628-2888-4964-b17a-0785d34355bf admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:57.985399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:57.986319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:57.986596 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:57.986867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] PATH_INFO: `/v3/projects/2f83ca7e7fc844f180ae057a576f3723/users/24c9257cf9254b82a2b9fa5ee41243be/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:58.004356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f83ca7e7fc844f180ae057a576f3723, user_id=24c9257cf9254b82a2b9fa5ee41243be, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:58.007078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:58.022116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1194/2395] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 21:59:57 2022] PUT /identity/v3/projects/2f83ca7e7fc844f180ae057a576f3723/users/24c9257cf9254b82a2b9fa5ee41243be/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 21:59:58.028107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83136b12-584c-4dbc-8523-f1e035d8a61f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:58.028510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83136b12-584c-4dbc-8523-f1e035d8a61f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:58.028861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83136b12-584c-4dbc-8523-f1e035d8a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:58.049330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-83136b12-584c-4dbc-8523-f1e035d8a61f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 21:59:58.050574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-83136b12-584c-4dbc-8523-f1e035d8a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:58.115272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1202/2396] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 21:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 21:59:58.125540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 21:59:58.127310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 21:59:58.189285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e56b9-5690-4cbb-b9ba-8ce3bb7a039c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 21:59:58.193029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 21:59:58.193716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 21:59:58.194046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 21:59:58.194323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 21:59:58.204210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 21:59:58.205655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 21:59:58.216700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1195/2397] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 21:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:00:03.144132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.144711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.145676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.180783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] MFA Rules not processed for user `24c9257cf9254b82a2b9fa5ee41243be`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:03.183012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:03.268671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1203/2398] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:00:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:00:03.274102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:03.289305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f085ff4a-32a4-4c08-83e7-765039e15043 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:03.295271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:03.296032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.296318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.296654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.297367 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:03.299422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:03.332173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1196/2399] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:00:03 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:00:03.332947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:03.345024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e51c24-cc0f-4a97-aade-a418df096b0f None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:03.406676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:03.407455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.407782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.408070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.408779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:03.410742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:03.432921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1204/2400] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:00:03 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:00:03.436646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:03.447684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8fd2-d7d9-4360-a843-9ad5158fd97b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:03.452406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:03.453084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.453365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.453639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.454873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:03.456016 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:03.471198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1197/2401] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:00:03.473239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:03.484021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:03.491674 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65693f31-4c3f-4d2e-849e-305ee531cf07 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:03.500473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:03.501853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.502277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.502700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] PATH_INFO: `/v3/projects/36901fd028e04a469089b3c5daa49bfd/users/9a88a11f1ac54e568d40216f068ec612/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.527245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36901fd028e04a469089b3c5daa49bfd, user_id=9a88a11f1ac54e568d40216f068ec612, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:03.531650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:03.551000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1205/2402] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:00:03 2022] PUT /identity/v3/projects/36901fd028e04a469089b3c5daa49bfd/users/9a88a11f1ac54e568d40216f068ec612/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:00:03.554964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:03.566751 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425743eb-221c-4d52-abc9-6d62e22fab1c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:03.627250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4af986-81e9-40ab-b687-2333864e1422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:03.627735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4af986-81e9-40ab-b687-2333864e1422 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.628027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4af986-81e9-40ab-b687-2333864e1422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.628305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4af986-81e9-40ab-b687-2333864e1422 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.629052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4af986-81e9-40ab-b687-2333864e1422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:03.630665 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4af986-81e9-40ab-b687-2333864e1422 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:03.641707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1198/2403] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:00:03.645787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:03.656920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd40f-a6ce-4931-97e1-43bb5ede2531 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:03.661602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931898b0-fc19-4fc5-b978-a85ab5d27da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:03.662505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931898b0-fc19-4fc5-b978-a85ab5d27da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:03.662786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931898b0-fc19-4fc5-b978-a85ab5d27da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:03.663122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931898b0-fc19-4fc5-b978-a85ab5d27da2 admin admin] PATH_INFO: `/v3/projects/36901fd028e04a469089b3c5daa49bfd/users/9a88a11f1ac54e568d40216f068ec612/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:03.665055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931898b0-fc19-4fc5-b978-a85ab5d27da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36901fd028e04a469089b3c5daa49bfd, user_id=9a88a11f1ac54e568d40216f068ec612, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:03.668586 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931898b0-fc19-4fc5-b978-a85ab5d27da2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:03.686502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1206/2404] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:00:03 2022] PUT /identity/v3/projects/36901fd028e04a469089b3c5daa49bfd/users/9a88a11f1ac54e568d40216f068ec612/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:00:09.432813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:09.433388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:09.433388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:09.463305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] MFA Rules not processed for user `9a88a11f1ac54e568d40216f068ec612`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:09.464669 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:09.511037 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:09.556913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1199/2405] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:00:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:00:09.570971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:09.572138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:09.572138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:09.577166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:09.578736 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:09.597629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:09.598863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:09.657502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1207/2406] 10.222.0.86 () {58 vars in 967 bytes} [Wed Jul 20 22:00:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:00:09.661780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150733fd-cb76-46e6-93d7-d0cbeb937fd2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:09.668757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a270e-e791-47aa-80f2-e00956c13266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:09.669422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a270e-e791-47aa-80f2-e00956c13266 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:09.669704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a270e-e791-47aa-80f2-e00956c13266 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:09.669983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a270e-e791-47aa-80f2-e00956c13266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:09.709154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:09.710566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:09.733961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a270e-e791-47aa-80f2-e00956c13266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:09.735582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a270e-e791-47aa-80f2-e00956c13266 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:09.744652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1200/2407] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:00:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:00:09.786272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b19afa5-7b6c-4380-9000-34740200bf36 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:09.792840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:09.793505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:09.793827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:09.794028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:09.808526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:09.810318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:09.824251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1208/2408] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 22:00:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 22:00:11.027811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:11.028280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:11.028365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:11.049503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] MFA Rules not processed for user `24c9257cf9254b82a2b9fa5ee41243be`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:11.050826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:11.059170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1201/2409] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:00:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:00:18.096442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:18.097959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:18.192347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291dfa7-af9b-41db-917f-3eedc86f5b9e service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:18.199942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:18.201312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:18.201594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:18.201930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] PATH_INFO: `/v3/users/030fcf8f4e654d08ac9c7e5dcb3eddd9` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:18.203338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=030fcf8f4e654d08ac9c7e5dcb3eddd9)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:18.205369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:18.208302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:18.262021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030fcf8f4e654d08ac9c7e5dcb3eddd9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.277065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030fcf8f4e654d08ac9c7e5dcb3eddd9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.284360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030fcf8f4e654d08ac9c7e5dcb3eddd9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.285083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030fcf8f4e654d08ac9c7e5dcb3eddd9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.287467 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030fcf8f4e654d08ac9c7e5dcb3eddd9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.304773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1209/2410] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:00:18 2022] DELETE /identity/v3/users/030fcf8f4e654d08ac9c7e5dcb3eddd9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:00:18.505442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:18.523872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a7769-44d2-4e0a-9599-40b1f3826aa5 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:18.580291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:18.587599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:18.588532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:18.588814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:18.589089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] PATH_INFO: `/v3/projects/c17214008914495b8083b74c6fc9e405` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:18.590375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c17214008914495b8083b74c6fc9e405)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:18.592339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:18.649074 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c17214008914495b8083b74c6fc9e405', 'request_id': 'req-bf91fc82-dfea-495b-934b-ae16a28cb59a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.656494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c17214008914495b8083b74c6fc9e405', 'request_id': 'req-bf91fc82-dfea-495b-934b-ae16a28cb59a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.662613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c17214008914495b8083b74c6fc9e405', 'request_id': 'req-bf91fc82-dfea-495b-934b-ae16a28cb59a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.676032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] The token cache is being invalidate because project c17214008914495b8083b74c6fc9e405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:00:18.676324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.676973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.678516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1202/2411] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:00:18 2022] DELETE /identity/v3/projects/c17214008914495b8083b74c6fc9e405 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:00:18.683564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:18.685140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:18.699130 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba38ce-3aa3-4a7e-a5f8-fee007aa3d34 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:18.758456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:18.759512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:18.759899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:18.760186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] PATH_INFO: `/v3/users/0a7cd440e1854db3906e6ba53e267496` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:18.761422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a7cd440e1854db3906e6ba53e267496)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:18.763430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:18.811735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a7cd440e1854db3906e6ba53e267496'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.824539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a7cd440e1854db3906e6ba53e267496'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.832643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a7cd440e1854db3906e6ba53e267496'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.833297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a7cd440e1854db3906e6ba53e267496'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.833893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-158e2b20-6ae1-44ec-be58-8c6a229db535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a7cd440e1854db3906e6ba53e267496'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:18.844716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:18.850139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1210/2412] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:00:18 2022] DELETE /identity/v3/users/0a7cd440e1854db3906e6ba53e267496 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:00:19.039235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:19.041558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:19.054744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91fc82-dfea-495b-934b-ae16a28cb59a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:19.116690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:19.117475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:19.117930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:19.118300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] PATH_INFO: `/v3/projects/3ee21db55ee449d7be269c7eaa58b878` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:19.119655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee21db55ee449d7be269c7eaa58b878)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:19.121691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:19.178709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee21db55ee449d7be269c7eaa58b878', 'request_id': 'req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:19.185218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee21db55ee449d7be269c7eaa58b878', 'request_id': 'req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:19.192002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee21db55ee449d7be269c7eaa58b878', 'request_id': 'req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:19.207180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] The token cache is being invalidate because project 3ee21db55ee449d7be269c7eaa58b878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:00:19.207460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:19.208311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-36f45ef4-20ec-4c5e-8079-dc0c56b8e521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:00:19.210086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1203/2413] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:00:19 2022] DELETE /identity/v3/projects/3ee21db55ee449d7be269c7eaa58b878 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:00:29.203501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=76,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:33.564303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:33.564861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:33.564861 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:33.594432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] MFA Rules not processed for user `c58389cb50e140d69f1d4c99d6d0e182`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:33.595775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:33.659636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1211/2414] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:00:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:00:38.835716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:38.836251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:38.836251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:38.872100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:38.873590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:38.978748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1204/2415] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 22:00:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:00:38.991338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:38.993559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:39.125826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d7d0f-6998-4633-a218-d67d9f6a81ed None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:39.130971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:39.133127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:39.133487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:39.134249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:39.144421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:39.145926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:39.156437 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1212/2416] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 22:00:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 22:00:48.971894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=41 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:49.152857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=72 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:00:49.690590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:49.695226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:49.902828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd87809-1d15-4e36-8f8d-560770f3bd41 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:49.908853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:49.909818 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:49.910133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:49.910392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:49.912371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:49.936732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:49.938771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df678fe6-a839-466b-b70c-8dad0527b904 service placement] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:49.963292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1205/2417] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:00:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:00:53.607049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:53.608541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:53.700227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4020d-c9e8-4ff2-bea0-b73db0cbc7d3 service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:53.704851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:53.704851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:53.705234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:53.705234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:53.707164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:53.810947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:53.812842 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:53.846146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1213/2418] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:00:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:00:59.520222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232d81-ad15-4ba9-a843-74a5403d74c2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:59.520764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232d81-ad15-4ba9-a843-74a5403d74c2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:59.520764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232d81-ad15-4ba9-a843-74a5403d74c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:59.540077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-b6232d81-ad15-4ba9-a843-74a5403d74c2 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:00:59.541305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b6232d81-ad15-4ba9-a843-74a5403d74c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:00:59.548639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1206/2419] 10.222.0.86 () {58 vars in 1009 bytes} [Wed Jul 20 22:00:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:00:59.562107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:00:59.577792 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995223b3-9f8f-49a4-82c8-d58ca3821dd3 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:00:59.580561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:00:59.581213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:00:59.581494 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:00:59.581767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:00:59.591416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:00:59.593014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:00:59.601589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1214/2420] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 22:00:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:01:09.533274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=44 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:09.598137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=65 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:18.297449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:18.298007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:18.298007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:18.316876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] MFA Rules not processed for user `d3d4323286624d928d94a6486d299eea`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:18.318099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:18.325577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1207/2421] 10.222.0.86 () {58 vars in 967 bytes} [Wed Jul 20 22:01:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:18.348418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:18.359655 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0c35f-e73a-4e90-a70e-a0cc012fd2a9 service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:18.365028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6952f7293b2f46f28941b48429dd2e73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:18.365717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:18.366002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:18.366281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:18.376310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:18.377941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:18.387309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1215/2422] 10.222.0.86 () {60 vars in 1384 bytes} [Wed Jul 20 22:01:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:01:28.311375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:28.383067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:30.653429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:30.654824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:30.666306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f796c7-a445-4e1e-88d6-0e5a90fe8eb6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:30.669854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:30.670485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:30.670762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:30.671031 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:30.672741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:30.764759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:30.766184 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:30.784739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1208/2423] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:01:33.627773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:33.629332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:33.645646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c25bfd-275f-44c1-8b73-a3763f82cc9c service swift] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:33.651505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:33.652238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:33.652592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:33.652970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:33.654937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:33.668763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:33.670457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:33.699445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:33.701028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:33.703611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1216/2424] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:01:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:01:33.732575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b619f-b1a5-40a8-bd9f-70e7d06af53a service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:33.737738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:33.738620 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:33.738902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:33.739225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] PATH_INFO: `/v3/users/60f7f43c067e4b819aeb488e9f09197a` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:33.740432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f7f43c067e4b819aeb488e9f09197a)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:33.742386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:33.807215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f7f43c067e4b819aeb488e9f09197a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:33.822942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f7f43c067e4b819aeb488e9f09197a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:33.832302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f7f43c067e4b819aeb488e9f09197a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:33.844366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f7f43c067e4b819aeb488e9f09197a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:33.845111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ecac3f58-b3e8-4814-a1e0-3749636e24b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f7f43c067e4b819aeb488e9f09197a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:33.857966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1209/2425] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:01:33 2022] DELETE /identity/v3/users/60f7f43c067e4b819aeb488e9f09197a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:01:34.059953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.072283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50cdb7-1516-408a-87a5-e34b38124678 service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.134934 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.135783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.136067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.136373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] PATH_INFO: `/v3/projects/e722bb68652c48f588d8b53351c6f2a6` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.137557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e722bb68652c48f588d8b53351c6f2a6)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.139518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.186164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e722bb68652c48f588d8b53351c6f2a6', 'request_id': 'req-97c8223a-8e90-4bed-9226-28fe78acd150'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:34.192175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e722bb68652c48f588d8b53351c6f2a6', 'request_id': 'req-97c8223a-8e90-4bed-9226-28fe78acd150'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:34.197962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e722bb68652c48f588d8b53351c6f2a6', 'request_id': 'req-97c8223a-8e90-4bed-9226-28fe78acd150'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:34.209881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] The token cache is being invalidate because project e722bb68652c48f588d8b53351c6f2a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:01:34.210103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:34.210725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-97c8223a-8e90-4bed-9226-28fe78acd150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:34.212901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1217/2426] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:01:34 2022] DELETE /identity/v3/projects/e722bb68652c48f588d8b53351c6f2a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:34.217918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.218210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.218492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.273591 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:34.278127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:34.280724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:34.340104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1210/2427] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:34.348173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.348465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.348778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.367473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:34.369143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:34.377343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1218/2428] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:34.381386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.391369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb271c-5b56-4381-b2c1-87e3d2ae0578 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.396083 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.396743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.397021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.397299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.398059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:01:34.398370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.400194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.411757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1211/2429] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:01:34.422040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.432391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd538c6-3205-4ca7-8cba-861c93a0d577 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.437011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.437662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.437940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.438209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.438893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.440835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.466506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1219/2430] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:01:34 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:01:34.470682 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.481381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5f49-7164-4af0-8397-b3eed67096a1 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.536835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.537512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.537792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.538077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.538777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.541440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.562235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1212/2431] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:01:34 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:01:34.566406 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.581675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75be67-5ccd-46c4-bb0e-dc3746f93204 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.586473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.587170 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.587455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.587728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.588472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.590066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.601343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1220/2432] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:01:34.606072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.616663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcb295-3f5a-4514-af75-0d98d1e1bae1 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.622228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff4362-86ce-4a9c-bd1e-0a450f0fbe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.622901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4362-86ce-4a9c-bd1e-0a450f0fbe1c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.623220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4362-86ce-4a9c-bd1e-0a450f0fbe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.623495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff4362-86ce-4a9c-bd1e-0a450f0fbe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.624232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff4362-86ce-4a9c-bd1e-0a450f0fbe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.626068 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff4362-86ce-4a9c-bd1e-0a450f0fbe1c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.643527 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1213/2433] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:01:34.647763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.658553 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093755b1-c79c-453f-9a39-86f6a4e621ea admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.663464 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.664940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.665218 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.665600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] PATH_INFO: `/v3/projects/bccef97639dd46819770dc3835f21e06/users/23feacc531be4ea98ac0ac98eb69853b/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.681948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccef97639dd46819770dc3835f21e06, user_id=23feacc531be4ea98ac0ac98eb69853b, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.685073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.699047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1221/2434] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:01:34 2022] PUT /identity/v3/projects/bccef97639dd46819770dc3835f21e06/users/23feacc531be4ea98ac0ac98eb69853b/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:34.709354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.709879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.709879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.740223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] MFA Rules not processed for user `23feacc531be4ea98ac0ac98eb69853b`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:34.741502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:34.809672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1214/2435] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:34.820356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:34.823042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:34.840908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:34.904492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5cc8f-e01b-4773-b473-9825a4abfb86 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:34.908522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd401a-932e-4ce2-94fc-1136e3441f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:34.909233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd401a-932e-4ce2-94fc-1136e3441f5e service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:34.909518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd401a-932e-4ce2-94fc-1136e3441f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:34.909796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd401a-932e-4ce2-94fc-1136e3441f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:34.920619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd401a-932e-4ce2-94fc-1136e3441f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:34.931360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd401a-932e-4ce2-94fc-1136e3441f5e service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:34.942038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1222/2436] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:01:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:01:43.473210 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:43.477480 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:43.538058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbaecb-f46d-474b-b633-62546fd6a8c0 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:43.542838 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:43.543755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:43.544040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:43.544315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] PATH_INFO: `/v3/users/23feacc531be4ea98ac0ac98eb69853b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:43.545515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] RBAC: Authorizing `identity:delete_user(user_id=23feacc531be4ea98ac0ac98eb69853b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:43.547565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:43.597784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23feacc531be4ea98ac0ac98eb69853b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:43.610595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23feacc531be4ea98ac0ac98eb69853b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:43.618429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23feacc531be4ea98ac0ac98eb69853b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:43.626897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23feacc531be4ea98ac0ac98eb69853b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:43.627552 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23feacc531be4ea98ac0ac98eb69853b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:43.634695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1215/2437] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:01:43 2022] DELETE /identity/v3/users/23feacc531be4ea98ac0ac98eb69853b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:43.639429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:43.639966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:43.639966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:43.659551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:43.660767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:43.724764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1223/2438] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:43.735304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:43.736738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:43.806612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8cca7-252d-4fd0-97ea-b40667d68afc admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:43.810410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb02b-f084-49bf-ad4a-c45caf3c7d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:43.811087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb02b-f084-49bf-ad4a-c45caf3c7d71 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:43.811379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb02b-f084-49bf-ad4a-c45caf3c7d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:43.811659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb02b-f084-49bf-ad4a-c45caf3c7d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:43.821490 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1eb02b-f084-49bf-ad4a-c45caf3c7d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:43.822936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1eb02b-f084-49bf-ad4a-c45caf3c7d71 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:43.833687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1216/2439] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:01:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:01:44.126940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.128457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:44.142855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d71b8-363f-4b12-8784-55cbe2dc14ff None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.148719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.149689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.149967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.150241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] PATH_INFO: `/v3/projects/bccef97639dd46819770dc3835f21e06` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.151488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccef97639dd46819770dc3835f21e06)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.153388 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.204945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccef97639dd46819770dc3835f21e06', 'request_id': 'req-0bcd84b9-6799-4729-87c4-d80d994589da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:44.211230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccef97639dd46819770dc3835f21e06', 'request_id': 'req-0bcd84b9-6799-4729-87c4-d80d994589da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:44.216975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccef97639dd46819770dc3835f21e06', 'request_id': 'req-0bcd84b9-6799-4729-87c4-d80d994589da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:44.232234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] The token cache is being invalidate because project bccef97639dd46819770dc3835f21e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:01:44.232513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:44.233217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0bcd84b9-6799-4729-87c4-d80d994589da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:01:44.234736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1224/2440] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:01:44 2022] DELETE /identity/v3/projects/bccef97639dd46819770dc3835f21e06 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:44.240365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.240717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.241089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.259720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:44.260942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:44.280460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:44.324499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1217/2441] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:44.333386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.333666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.333945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.354377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:44.355656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:44.363747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1225/2442] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:44.368311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.379390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78e27f-414c-4fcd-aea9-4fceaa072d66 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.385134 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.385807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.386084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.386359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.387126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:01:44.387434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.389053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.402366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1218/2443] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:01:44.406729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.417002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcf501-ebee-4345-9ab2-dcf47466e3c1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.421727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.422420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.422698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.422968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.423733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.425687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.437622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:44.460061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1226/2444] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/projects => generated 353 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:01:44.462042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.472419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c342790-9de5-4a92-8f78-a8a0c742677b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.531441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.532118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.532284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.532566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.533274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.535205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.556071 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1219/2445] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/users => generated 277 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:01:44.563544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.576180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352da3ed-32fd-489c-986a-01f95d5c52c2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.580860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.581523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.581801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.582076 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.582796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.584429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.595927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1227/2446] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:01:44.600017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.611555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1182fbb-a637-411c-9c97-4d73dcbade9e admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.616430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da693f-b88d-4a33-b4a8-7fd30da700fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.618431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da693f-b88d-4a33-b4a8-7fd30da700fc admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.618431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da693f-b88d-4a33-b4a8-7fd30da700fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.618431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da693f-b88d-4a33-b4a8-7fd30da700fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.618883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da693f-b88d-4a33-b4a8-7fd30da700fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.620531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da693f-b88d-4a33-b4a8-7fd30da700fc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.631442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1220/2447] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:01:44.636118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.647087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cbc69-fcb1-41b8-87fc-1a6b22bb754c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.657748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.658661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.658942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.659266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] PATH_INFO: `/v3/projects/b10b587d145441de895dfb2ba6929301/users/3f7e161f7f2e4c2ca76c6bf84e01a0b2/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.675810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10b587d145441de895dfb2ba6929301, user_id=3f7e161f7f2e4c2ca76c6bf84e01a0b2, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.678555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.692585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1228/2448] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:01:44 2022] PUT /identity/v3/projects/b10b587d145441de895dfb2ba6929301/users/3f7e161f7f2e4c2ca76c6bf84e01a0b2/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:44.703635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.703919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.704193 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.733244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] MFA Rules not processed for user `3f7e161f7f2e4c2ca76c6bf84e01a0b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:44.734470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:44.802179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1221/2449] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:44.806762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.872504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a5ed8-0881-46cb-a2ee-9f35095072d5 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.877579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.878243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.878526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.878808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.879551 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.881469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:44.909039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1229/2450] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:01:44.913044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:44.924477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de115fa2-768d-48d7-b8f3-3a619e726064 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:44.942195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:44.985741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:44.986422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:44.986708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:44.986984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:44.987723 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:44.989644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.011499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1222/2451] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:01:44 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:01:45.015671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:45.026239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8689a8be-e1e4-4054-a45e-8b3dee615c25 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:45.030907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:45.031592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.031871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.032156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.032879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:45.034482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.046527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1230/2452] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:01:45.050864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:45.071458 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b202dc2-0d70-4e68-837e-d80afad54223 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:45.076912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:45.077833 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.078111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.078378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] PATH_INFO: `/v3/projects/f5d8fc0fab1f447f97c60291801384a7/users/735da566aa8140e29d5c3a0976c900a2/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.094470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d8fc0fab1f447f97c60291801384a7, user_id=735da566aa8140e29d5c3a0976c900a2, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:45.097196 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.119356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1223/2453] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:01:45 2022] PUT /identity/v3/projects/f5d8fc0fab1f447f97c60291801384a7/users/735da566aa8140e29d5c3a0976c900a2/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:45.123364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:45.134526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85913f4-af9d-4546-9bd0-3a09e0593a1e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:45.195906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2c7b0-6f77-4e34-a554-1816a26c23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:45.196573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2c7b0-6f77-4e34-a554-1816a26c23c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.196850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2c7b0-6f77-4e34-a554-1816a26c23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.197128 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2c7b0-6f77-4e34-a554-1816a26c23c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.197849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2c7b0-6f77-4e34-a554-1816a26c23c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:45.199492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2c7b0-6f77-4e34-a554-1816a26c23c1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.209973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1231/2454] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:01:45.214753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:45.225338 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ad183-80ca-4ee5-ab87-b9d748cc5fab admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:45.231033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:45.231967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.232251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.232549 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] PATH_INFO: `/v3/projects/f5d8fc0fab1f447f97c60291801384a7/users/735da566aa8140e29d5c3a0976c900a2/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.234579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5d8fc0fab1f447f97c60291801384a7, user_id=735da566aa8140e29d5c3a0976c900a2, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:45.237291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.248502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1224/2455] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:01:45 2022] PUT /identity/v3/projects/f5d8fc0fab1f447f97c60291801384a7/users/735da566aa8140e29d5c3a0976c900a2/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:01:45.260254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.260623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.260968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.289893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] MFA Rules not processed for user `735da566aa8140e29d5c3a0976c900a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:01:45.291139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:45.305394 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:45.371543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1232/2456] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:01:45 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:01:45.383466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:45.385764 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:01:45.448744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdaf014-7927-4960-abef-544581653bf9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:45.452434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:45.453116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.453399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.453677 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.464250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:45.465724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.475732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1225/2457] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:01:45 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:01:45.555951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:01:45.566093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93ec0a-0288-458e-9f7d-aad1d45352b5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:01:45.569777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc40e37-51a8-461e-b701-ae094b29e564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:01:45.571090 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc40e37-51a8-461e-b701-ae094b29e564 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:01:45.571369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc40e37-51a8-461e-b701-ae094b29e564 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:01:45.571708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc40e37-51a8-461e-b701-ae094b29e564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:01:45.631873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc40e37-51a8-461e-b701-ae094b29e564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:01:45.633835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc40e37-51a8-461e-b701-ae094b29e564 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:01:45.643481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1233/2458] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:01:45 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:01:55.471226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:01:55.639030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=50 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:00.790421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:00.866913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb52fd-fa58-48e4-b34e-93c185f0e063 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:00.872459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:00.873354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:00.873639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:00.873913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] PATH_INFO: `/v3/users/3f7e161f7f2e4c2ca76c6bf84e01a0b2` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:00.875344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7e161f7f2e4c2ca76c6bf84e01a0b2)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:00.877594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:00.928291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7e161f7f2e4c2ca76c6bf84e01a0b2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:00.940694 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7e161f7f2e4c2ca76c6bf84e01a0b2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:00.947895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7e161f7f2e4c2ca76c6bf84e01a0b2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:00.956891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7e161f7f2e4c2ca76c6bf84e01a0b2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:00.957521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7e161f7f2e4c2ca76c6bf84e01a0b2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:00.964625 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1226/2459] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:00 2022] DELETE /identity/v3/users/3f7e161f7f2e4c2ca76c6bf84e01a0b2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:00.969247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:00.969809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:00.969985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:00.988678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:00.990107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:01.057537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1234/2460] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:01.068534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:01.069996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:01.130998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f8813-1033-455f-bb8c-243ddd870b9f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:01.134646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:01.135352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:01.135636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:01.135912 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:01.145331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:01.146768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:01.156991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1227/2461] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:01.460314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:01.462342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:01.478163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0e653-0811-4ce7-8f1d-886cfc5f8d40 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:01.483886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:01.484472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:01.484856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:01.485232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] PATH_INFO: `/v3/projects/b10b587d145441de895dfb2ba6929301` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:01.486592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10b587d145441de895dfb2ba6929301)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:01.488700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:01.544250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10b587d145441de895dfb2ba6929301', 'request_id': 'req-979542f8-5509-4dee-b7c8-14aa7338bd18'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.551269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10b587d145441de895dfb2ba6929301', 'request_id': 'req-979542f8-5509-4dee-b7c8-14aa7338bd18'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.557967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10b587d145441de895dfb2ba6929301', 'request_id': 'req-979542f8-5509-4dee-b7c8-14aa7338bd18'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.571277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] The token cache is being invalidate because project b10b587d145441de895dfb2ba6929301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:01.571661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.572274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.573857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1235/2462] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:01 2022] DELETE /identity/v3/projects/b10b587d145441de895dfb2ba6929301 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:01.578068 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:01.588864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997ebc3-4056-412a-9be2-29a9a1e38c33 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:01.609223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:01.658326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:01.659150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:01.659431 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:01.659703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] PATH_INFO: `/v3/users/735da566aa8140e29d5c3a0976c900a2` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:01.660910 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] RBAC: Authorizing `identity:delete_user(user_id=735da566aa8140e29d5c3a0976c900a2)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:01.662846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:01.712353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '735da566aa8140e29d5c3a0976c900a2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.725022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '735da566aa8140e29d5c3a0976c900a2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.732709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '735da566aa8140e29d5c3a0976c900a2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.743393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '735da566aa8140e29d5c3a0976c900a2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.744824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-cd7266b4-c4e4-4f92-a256-25c633c2b65b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '735da566aa8140e29d5c3a0976c900a2'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:01.751970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1228/2463] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:01 2022] DELETE /identity/v3/users/735da566aa8140e29d5c3a0976c900a2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:02.076749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.084681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:02.111307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979542f8-5509-4dee-b7c8-14aa7338bd18 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.184236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.185109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.185396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.185667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] PATH_INFO: `/v3/projects/f5d8fc0fab1f447f97c60291801384a7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.186913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5d8fc0fab1f447f97c60291801384a7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.188927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.223760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:02.244108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5d8fc0fab1f447f97c60291801384a7', 'request_id': 'req-c48d49fa-37d4-4278-b0d6-602b65182034'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:02.250908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5d8fc0fab1f447f97c60291801384a7', 'request_id': 'req-c48d49fa-37d4-4278-b0d6-602b65182034'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:02.256724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5d8fc0fab1f447f97c60291801384a7', 'request_id': 'req-c48d49fa-37d4-4278-b0d6-602b65182034'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:02.270882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] The token cache is being invalidate because project f5d8fc0fab1f447f97c60291801384a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:02.271225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:02.271865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c48d49fa-37d4-4278-b0d6-602b65182034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:02.273427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1236/2464] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:02 2022] DELETE /identity/v3/projects/f5d8fc0fab1f447f97c60291801384a7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:02.278768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.279545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.279545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.298100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:02.299582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:02.392741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1229/2465] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:02.401183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.401567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.401912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.421015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:02.422322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:02.432302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1237/2466] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:02.437169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.450106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c957f1-2de2-4aee-afdb-b53ec1582174 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.455485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.456149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.456508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.456828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.457574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:02.458080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.461434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.475456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1230/2467] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:02.480957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.491258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16d79d-caf0-4ad7-87e6-5437e8ce6913 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.501230 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.501930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.502211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.502508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.503245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.505154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.532327 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1238/2468] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:02.536535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.546824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29799b24-b3f3-4ca3-9eec-145ccac9b45c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.574028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:02.607747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.608478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.608817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.609100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.609791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.611798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.634443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1231/2469] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:02.639470 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.653400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8725b-5a7d-4ce7-b636-f37f5daab723 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.658767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.659477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.659760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.660037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.660762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.662378 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.673242 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1239/2470] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:02.678158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.692573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff31249-da6e-49a2-b3c2-5c09854a061a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.698595 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6ec8a-4c12-44bf-8d23-7579344c55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.699687 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ec8a-4c12-44bf-8d23-7579344c55a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.700177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ec8a-4c12-44bf-8d23-7579344c55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.700540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6ec8a-4c12-44bf-8d23-7579344c55a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.701536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6ec8a-4c12-44bf-8d23-7579344c55a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.703546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6ec8a-4c12-44bf-8d23-7579344c55a4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.716779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1232/2471] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:02.721439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.734202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b8b2a-bbae-4d53-a57b-46d791d2169a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.738952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.739884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.740166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.740441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] PATH_INFO: `/v3/projects/f7c46219ee454c58ac4b968b5d494aee/users/56cd2def5b7f47d4a15ea3e19d8d4191/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.756818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c46219ee454c58ac4b968b5d494aee, user_id=56cd2def5b7f47d4a15ea3e19d8d4191, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.760301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.776257 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1240/2472] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:02 2022] PUT /identity/v3/projects/f7c46219ee454c58ac4b968b5d494aee/users/56cd2def5b7f47d4a15ea3e19d8d4191/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:02.786908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.787252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.787519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.815489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] MFA Rules not processed for user `56cd2def5b7f47d4a15ea3e19d8d4191`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:02.816714 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:02.885794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1233/2473] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:02.890349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:02.952690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d1c6f-d809-40cb-9666-397bc27099aa admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:02.957369 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:02.958037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:02.958322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:02.958598 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:02.959340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:02.961997 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:02.989045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1241/2474] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:02.993321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.004534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23d6e4e-ed46-4f42-a63f-d67aea17efba None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.067239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.067911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.068221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.068503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.069240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.071927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.092828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1234/2475] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:02 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:03.096841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.107760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46455bb9-8973-45fc-904e-d1de9e79c442 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.112487 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.113155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.113443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.113722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.114460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.116108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.127788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1242/2476] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:03.132248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.143453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016a67b-4ef6-42cf-905c-86a5faddb301 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.148287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.149494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.149780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.150061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] PATH_INFO: `/v3/projects/dbc5ce482ae3470983a3967871a4c314/users/a94127dcd5e742168212deaaf544bb3e/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.167791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc5ce482ae3470983a3967871a4c314, user_id=a94127dcd5e742168212deaaf544bb3e, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.170466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.184601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1235/2477] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:03 2022] PUT /identity/v3/projects/dbc5ce482ae3470983a3967871a4c314/users/a94127dcd5e742168212deaaf544bb3e/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:03.188613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.200391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541286c-4be7-44fe-a7d9-43ac37353779 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.221281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:03.263930 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643e549-f820-455f-91fd-51c8f5712016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.264684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643e549-f820-455f-91fd-51c8f5712016 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.265175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643e549-f820-455f-91fd-51c8f5712016 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.265576 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643e549-f820-455f-91fd-51c8f5712016 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.266323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643e549-f820-455f-91fd-51c8f5712016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.267981 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643e549-f820-455f-91fd-51c8f5712016 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.280158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1243/2478] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:03.284641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.291450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:03.298958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90747175-5838-4460-a716-0f1331cf0ee4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.303854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.304744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.305108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.305568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] PATH_INFO: `/v3/projects/dbc5ce482ae3470983a3967871a4c314/users/a94127dcd5e742168212deaaf544bb3e/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.307738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbc5ce482ae3470983a3967871a4c314, user_id=a94127dcd5e742168212deaaf544bb3e, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.314403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.323840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1236/2479] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:03 2022] PUT /identity/v3/projects/dbc5ce482ae3470983a3967871a4c314/users/a94127dcd5e742168212deaaf544bb3e/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:03.335790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.336615 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.336940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.364580 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] MFA Rules not processed for user `a94127dcd5e742168212deaaf544bb3e`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:03.365821 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:03.435933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1244/2480] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:03.447532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.449265 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:03.519344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f49471-a4eb-4cce-bbce-d7fb6b25b485 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.524268 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.524981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.525273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.525556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.536029 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.537495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.548634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1237/2481] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:03.637180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:03.639440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:03.654649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7336ad-0ec7-44ea-ba16-004d6d34cdc3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:03.658566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:03.659461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:03.659815 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:03.660099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:03.736410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:03.737868 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:03.747085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1245/2482] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:06.088699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:06.090125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:06.149044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048bcce-d7ad-41f6-a234-fd85bd303755 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:06.152729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:06.153378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:06.153660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:06.153935 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:06.155750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:06.231809 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:06.233273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23eb3d2-ee4d-41e4-b154-5cccb167f772 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:06.245696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1238/2483] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:08.630305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:08.652643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac5183-9be0-4e42-9ef9-479af128556e service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:08.660647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:08.661817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:08.662097 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:08.662374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] PATH_INFO: `/v3/users/56cd2def5b7f47d4a15ea3e19d8d4191` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:08.663666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56cd2def5b7f47d4a15ea3e19d8d4191)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:08.665786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:08.708623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:08.717080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56cd2def5b7f47d4a15ea3e19d8d4191'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:08.731293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56cd2def5b7f47d4a15ea3e19d8d4191'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:08.739515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56cd2def5b7f47d4a15ea3e19d8d4191'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:08.740271 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56cd2def5b7f47d4a15ea3e19d8d4191'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:08.740875 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56cd2def5b7f47d4a15ea3e19d8d4191'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:08.762401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1246/2484] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:08 2022] DELETE /identity/v3/users/56cd2def5b7f47d4a15ea3e19d8d4191 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:08.767190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:08.767729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:08.767849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:08.786327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:08.787589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:08.841522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:08.864988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1239/2485] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:08.876961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:08.878395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:08.951972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b5f56-675f-41d0-827d-c1c625aebf33 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:08.957408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:08.958513 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:08.958802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:08.959106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:08.969506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:08.971368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:08.982991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1247/2486] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:09.487409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:09.488740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:09.502499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb6df5-9fbc-49e4-a2dc-90c2583d0256 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:09.508888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:09.509498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:09.510167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:09.510478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] PATH_INFO: `/v3/projects/f7c46219ee454c58ac4b968b5d494aee` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:09.513642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c46219ee454c58ac4b968b5d494aee)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:09.515782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:09.599222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c46219ee454c58ac4b968b5d494aee', 'request_id': 'req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.612356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c46219ee454c58ac4b968b5d494aee', 'request_id': 'req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.620590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c46219ee454c58ac4b968b5d494aee', 'request_id': 'req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.641760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] The token cache is being invalidate because project f7c46219ee454c58ac4b968b5d494aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:09.641962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.642608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.646036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1240/2487] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:09 2022] DELETE /identity/v3/projects/f7c46219ee454c58ac4b968b5d494aee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:09.650713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:09.661746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287122-0b56-412e-86e0-a3e4f6eae5a7 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:09.750682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:09.751820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:09.752173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:09.752569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] PATH_INFO: `/v3/users/a94127dcd5e742168212deaaf544bb3e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:09.753940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94127dcd5e742168212deaaf544bb3e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:09.756092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:09.822092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94127dcd5e742168212deaaf544bb3e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.842948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:09.845779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94127dcd5e742168212deaaf544bb3e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.866202 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94127dcd5e742168212deaaf544bb3e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.867786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94127dcd5e742168212deaaf544bb3e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.869803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94127dcd5e742168212deaaf544bb3e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:09.886312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1248/2488] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:09 2022] DELETE /identity/v3/users/a94127dcd5e742168212deaaf544bb3e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:10.086738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.090973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.102716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8f805-9c1e-4653-9dfd-592f0684b3c8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.168381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.169291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.169716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.169985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] PATH_INFO: `/v3/users/c58389cb50e140d69f1d4c99d6d0e182` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.171240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58389cb50e140d69f1d4c99d6d0e182)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.173652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.205336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.207144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.230218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58389cb50e140d69f1d4c99d6d0e182'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.233515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbae08-86e7-48c5-90cd-e70f658fa53e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.239108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.240126 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.240626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.240908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] PATH_INFO: `/v3/projects/dbc5ce482ae3470983a3967871a4c314` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.242266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbc5ce482ae3470983a3967871a4c314)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.244248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.248023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58389cb50e140d69f1d4c99d6d0e182'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.250080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:10.258288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58389cb50e140d69f1d4c99d6d0e182'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.266695 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58389cb50e140d69f1d4c99d6d0e182'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.267417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a06e024b-4495-48bc-850a-402222bff6a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58389cb50e140d69f1d4c99d6d0e182'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.274792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1241/2489] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:10 2022] DELETE /identity/v3/users/c58389cb50e140d69f1d4c99d6d0e182 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:10.298783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbc5ce482ae3470983a3967871a4c314', 'request_id': 'req-c101d542-e682-41e8-ad63-0cc28b70edb9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.306062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbc5ce482ae3470983a3967871a4c314', 'request_id': 'req-c101d542-e682-41e8-ad63-0cc28b70edb9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.312070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbc5ce482ae3470983a3967871a4c314', 'request_id': 'req-c101d542-e682-41e8-ad63-0cc28b70edb9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.326984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] The token cache is being invalidate because project dbc5ce482ae3470983a3967871a4c314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:10.327307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.327922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-c101d542-e682-41e8-ad63-0cc28b70edb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.329452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1249/2490] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:10 2022] DELETE /identity/v3/projects/dbc5ce482ae3470983a3967871a4c314 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:10.334382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.334668 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.334940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.353390 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:10.354623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.417326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1242/2491] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:10.425101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.425386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.425662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.447684 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:10.448820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.458318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1250/2492] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:10.462368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.462368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.463708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.475673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe9f9a3-da1b-4fb1-a1cd-a3dd7fa89ec6 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.478733 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed7df0-4f51-4ed2-b8ce-4e6326dc12cf None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.482532 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.483554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.483843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.484114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] PATH_INFO: `/v3/projects/a41b592357304d8787cc8e1ab4c9df15` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.484794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.485471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.485605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41b592357304d8787cc8e1ab4c9df15)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.485951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.486277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.487500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.488059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:10.488540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.491136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.506397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1251/2493] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:10.511440 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.521771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f89cf1-ce2a-4621-919e-8b2bbb740251 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.528066 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.528974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.529264 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.529544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.530246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.532275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.544763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41b592357304d8787cc8e1ab4c9df15', 'request_id': 'req-fbfee8d7-d73c-46b6-8b65-85ef2130a420'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.551807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41b592357304d8787cc8e1ab4c9df15', 'request_id': 'req-fbfee8d7-d73c-46b6-8b65-85ef2130a420'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.558323 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41b592357304d8787cc8e1ab4c9df15', 'request_id': 'req-fbfee8d7-d73c-46b6-8b65-85ef2130a420'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.560933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1252/2494] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:10.565468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.572308 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] The token cache is being invalidate because project a41b592357304d8787cc8e1ab4c9df15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:10.572717 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.573566 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fbfee8d7-d73c-46b6-8b65-85ef2130a420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:10.575631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1243/2495] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:10 2022] DELETE /identity/v3/projects/a41b592357304d8787cc8e1ab4c9df15 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:10.581169 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179ccc5-8035-412a-9d5a-79f0fb36b625 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.581460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179ccc5-8035-412a-9d5a-79f0fb36b625 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.581742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179ccc5-8035-412a-9d5a-79f0fb36b625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.602078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e179ccc5-8035-412a-9d5a-79f0fb36b625 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:10.603381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e179ccc5-8035-412a-9d5a-79f0fb36b625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.641773 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe3be7-1067-4fd6-aad8-1b6fc89788b2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.646640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.647364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.647638 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.647908 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.649224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.651279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.671430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1244/2496] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:10.687362 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1253/2497] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:10.689042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.689440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.689860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.693379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.696563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.713933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a344167-822a-4928-90f8-a63712dd7838 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.715792 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:10.717026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.718687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.719411 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.719694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.719970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.720706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.722334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.725018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=9,keystone:SELECT=85,keystone:INSERT=6 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:10.725632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1245/2498] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:10.730630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.735492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:10.739636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1254/2499] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:10.740901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.741430 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da387f2-08cd-470c-b64a-65efeff63834 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.748280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.749077 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.749590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.750028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.751255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:10.751570 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.752628 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdaf1e-ac78-41f0-bba2-15274b65d9d8 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.753260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.757281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.757974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.758255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.758561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.759323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.760917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.765755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1246/2500] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:02:10.770089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.771884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1255/2501] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:10.776364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.781181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201073e1-0fdd-43b5-8035-2536dc1c984c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.786088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2adc2-5fb8-47ae-bd78-cd6ee4986141 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.786741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.787433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.787719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.788001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.789459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.791572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.791709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb9265-50ec-46ad-939b-a723b2ac630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.792499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb9265-50ec-46ad-939b-a723b2ac630f admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.792780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb9265-50ec-46ad-939b-a723b2ac630f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.793176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb9265-50ec-46ad-939b-a723b2ac630f admin admin] PATH_INFO: `/v3/projects/0e03b60adff748099066f2c2e4f0ea80/users/00500be23cf34120a1308790322ddb0b/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.808209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb9265-50ec-46ad-939b-a723b2ac630f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e03b60adff748099066f2c2e4f0ea80, user_id=00500be23cf34120a1308790322ddb0b, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.811244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb9265-50ec-46ad-939b-a723b2ac630f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.817816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1247/2502] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:10.822266 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.824689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1256/2503] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:10 2022] PUT /identity/v3/projects/0e03b60adff748099066f2c2e4f0ea80/users/00500be23cf34120a1308790322ddb0b/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:10.836595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.836882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.837164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.863787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] MFA Rules not processed for user `00500be23cf34120a1308790322ddb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:10.865011 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.889366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb8182-4154-4e59-9428-3d46c3c07394 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.931647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1257/2504] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:10.943749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.945823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:10.951280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.951949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.952230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.952529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.953211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.955180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:10.975945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1248/2505] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:10 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:10.980955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:10.990858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fed75-da4b-40fd-a2ef-ffe84dd39004 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:10.995654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:10.996305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:10.996585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:10.996858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:10.997746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:10.999365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.006259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22e1f-a26f-42e6-9897-189b4ab1d96c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.010647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.014111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1249/2506] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:11.014245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.014245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.014245 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.015146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.020892 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.022336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.025189 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89099502-8666-424f-9adb-3ca9f1369c03 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.029878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.030524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.030807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.031105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.031661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1258/2507] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:11.031956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.033619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.048022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1250/2508] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:11.049441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.050941 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.060959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b449b-46a1-4845-ac95-10ffe16c51fd service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.061132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69be02-fe5c-459d-80f0-98f5e864337b admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.065585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4662a1-b2cc-482f-97f0-ce928411e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.065727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.066472 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4662a1-b2cc-482f-97f0-ce928411e862 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.066593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.066750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4662a1-b2cc-482f-97f0-ce928411e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.066863 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.067030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4662a1-b2cc-482f-97f0-ce928411e862 admin admin] PATH_INFO: `/v3/projects/d8d189cc396346bb91dbaf2af29b747a/users/2cacbaf0e55a4b8ea0dd0fddd07db7d9/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.067155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] PATH_INFO: `/v3/users/00500be23cf34120a1308790322ddb0b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.068291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00500be23cf34120a1308790322ddb0b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.070264 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.084082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4662a1-b2cc-482f-97f0-ce928411e862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d189cc396346bb91dbaf2af29b747a, user_id=2cacbaf0e55a4b8ea0dd0fddd07db7d9, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.087316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4662a1-b2cc-482f-97f0-ce928411e862 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.102129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1259/2509] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:11 2022] PUT /identity/v3/projects/d8d189cc396346bb91dbaf2af29b747a/users/2cacbaf0e55a4b8ea0dd0fddd07db7d9/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:11.112948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.113319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.113789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.128307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:11.138253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00500be23cf34120a1308790322ddb0b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.143916 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] MFA Rules not processed for user `2cacbaf0e55a4b8ea0dd0fddd07db7d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:11.145200 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.151371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00500be23cf34120a1308790322ddb0b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.158306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00500be23cf34120a1308790322ddb0b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.164828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:11.168660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00500be23cf34120a1308790322ddb0b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.169320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-344837f5-4b9d-40e9-867f-a69b1ec6eab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00500be23cf34120a1308790322ddb0b'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.176474 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1251/2510] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:11 2022] DELETE /identity/v3/users/00500be23cf34120a1308790322ddb0b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:11.182125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.182457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.182741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.202171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:11.203804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.225837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1260/2511] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:11.238312 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.239713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.270740 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1252/2512] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:11.284109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.285658 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.354433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63b601-9ef2-4b75-80f3-6f8c0eb05b30 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.362483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.362483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.362483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.362483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.364779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba86e5d-7378-4151-b5b4-7d55edb06f53 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.374176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.376118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.388302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1253/2513] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:11.442549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.443537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.443917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.444387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.491720 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=100 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:11.518220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.520061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.531475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1261/2514] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:11.750337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.752295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.766021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b2f52-d2f2-4611-9498-1ce25594e32f service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.785893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:11.787765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.792539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.793547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.793897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.794239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] PATH_INFO: `/v3/projects/0e03b60adff748099066f2c2e4f0ea80` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.795711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e03b60adff748099066f2c2e4f0ea80)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.797994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.802856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f29dc-38c0-44c1-9b97-c16f0ac5d183 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:11.808772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:11.809649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.809924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.810301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] PATH_INFO: `/v3/users/2cacbaf0e55a4b8ea0dd0fddd07db7d9` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.811568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cacbaf0e55a4b8ea0dd0fddd07db7d9)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:11.813601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:11.856322 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e03b60adff748099066f2c2e4f0ea80', 'request_id': 'req-60535d21-6492-41e9-9b4a-e4ed5364baa3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.858503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:11.864617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e03b60adff748099066f2c2e4f0ea80', 'request_id': 'req-60535d21-6492-41e9-9b4a-e4ed5364baa3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.872547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e03b60adff748099066f2c2e4f0ea80', 'request_id': 'req-60535d21-6492-41e9-9b4a-e4ed5364baa3'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.876042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cacbaf0e55a4b8ea0dd0fddd07db7d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.886580 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] The token cache is being invalidate because project 0e03b60adff748099066f2c2e4f0ea80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:11.886911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.887548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-60535d21-6492-41e9-9b4a-e4ed5364baa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.889236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1254/2515] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:11 2022] DELETE /identity/v3/projects/0e03b60adff748099066f2c2e4f0ea80 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:11.891168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cacbaf0e55a4b8ea0dd0fddd07db7d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.894787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab85a-03f2-46c2-be5a-370769bd1e32 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.895113 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab85a-03f2-46c2-be5a-370769bd1e32 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.895396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edab85a-03f2-46c2-be5a-370769bd1e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.900280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cacbaf0e55a4b8ea0dd0fddd07db7d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.901091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cacbaf0e55a4b8ea0dd0fddd07db7d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.902081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e4e9a43-d93c-45b6-84ca-5d96ecd99024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cacbaf0e55a4b8ea0dd0fddd07db7d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:11.922311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1262/2516] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:11 2022] DELETE /identity/v3/users/2cacbaf0e55a4b8ea0dd0fddd07db7d9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:11.923956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-7edab85a-03f2-46c2-be5a-370769bd1e32 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:11.925515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7edab85a-03f2-46c2-be5a-370769bd1e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.926757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.927206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.927508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:11.944757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:11.946914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:11.989955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1255/2517] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:11.998131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:11.998575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:11.998856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.010179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1263/2518] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:12.017593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:12.018814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.019973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.021313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.027514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1256/2519] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:12.031607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.041033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08dca6-9e38-4ede-b765-e828f77a15a3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.045851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.046617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.047026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.047422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.048267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:12.048675 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.050354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.065748 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1257/2520] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:02:12.066779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.077249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b221d9-c3eb-4805-afb2-3a485ea2dede admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.080972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:12.082082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.083239 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.083349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.083680 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.084453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.087763 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.093806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2e7d3-55c1-49af-b699-c1eb78d818e3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.098681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.099410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.099693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.099972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.110320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.111811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.119949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1258/2521] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:12.121006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.126336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1264/2522] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:12.188392 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0feccbd-23c2-4762-b876-91b727af74c1 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.193602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.194290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.194577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.194884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.195684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.197919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.220333 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1259/2523] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:12.224738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.236220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb20f1-1eec-473a-ad10-78e904a70148 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.241222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.242436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.242845 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.243152 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.243927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.245641 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.256637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1265/2524] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:12.261328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.271800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56830f3-4da3-458c-93eb-52770a0ec679 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.293463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea2c34-a069-4ac1-9711-037aec99f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.294161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea2c34-a069-4ac1-9711-037aec99f862 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.294444 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea2c34-a069-4ac1-9711-037aec99f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.294720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea2c34-a069-4ac1-9711-037aec99f862 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.295498 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea2c34-a069-4ac1-9711-037aec99f862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.297099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea2c34-a069-4ac1-9711-037aec99f862 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.310487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1260/2525] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:12.311739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.321812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b19a1-a295-431d-bd60-ab97251907f6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.326803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.327800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.328148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.328506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] PATH_INFO: `/v3/projects/dfc3da30c23e471581b7c3742e5f6284/users/ec1e4f6f7b7549b297e9dcc1e923fb3a/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.344326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfc3da30c23e471581b7c3742e5f6284, user_id=ec1e4f6f7b7549b297e9dcc1e923fb3a, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.347013 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.361134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1266/2526] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:12 2022] PUT /identity/v3/projects/dfc3da30c23e471581b7c3742e5f6284/users/ec1e4f6f7b7549b297e9dcc1e923fb3a/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:12.372126 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.372425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.372707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.402768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] MFA Rules not processed for user `ec1e4f6f7b7549b297e9dcc1e923fb3a`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:12.404334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.404488 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.406401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.410929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:12.480066 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1261/2527] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:12.487006 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1c818-3b42-489d-aa23-e5b6e61b6277 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.491300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.492062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.492963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.493068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.493299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.493573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] PATH_INFO: `/v3/projects/d8d189cc396346bb91dbaf2af29b747a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.494782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d189cc396346bb91dbaf2af29b747a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.496751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.551032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d189cc396346bb91dbaf2af29b747a', 'request_id': 'req-293825d0-85ae-401a-9078-ead052454161'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:12.558300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d189cc396346bb91dbaf2af29b747a', 'request_id': 'req-293825d0-85ae-401a-9078-ead052454161'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:12.559075 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e0270-5222-4a67-b4fb-c5aed2e520ca None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.563941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d189cc396346bb91dbaf2af29b747a', 'request_id': 'req-293825d0-85ae-401a-9078-ead052454161'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:12.578181 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] The token cache is being invalidate because project d8d189cc396346bb91dbaf2af29b747a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:12.578468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:12.579139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-293825d0-85ae-401a-9078-ead052454161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:12.580847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1267/2528] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:12 2022] DELETE /identity/v3/projects/d8d189cc396346bb91dbaf2af29b747a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:12.586167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0d0a-d51f-4dbb-abfa-5d88ce55a916 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.586460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0d0a-d51f-4dbb-abfa-5d88ce55a916 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.587018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0d0a-d51f-4dbb-abfa-5d88ce55a916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.608597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4f6d0d0a-d51f-4dbb-abfa-5d88ce55a916 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:12.609850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4f6d0d0a-d51f-4dbb-abfa-5d88ce55a916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.616887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.618132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.618415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.618747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.620638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.655616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:12.678114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1268/2529] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:12.683377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.684876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.686458 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b621d-7936-4463-9610-300e4631064c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.686744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b621d-7936-4463-9610-300e4631064c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.687019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b621d-7936-4463-9610-300e4631064c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.694313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1262/2530] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:12.706485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-749b621d-7936-4463-9610-300e4631064c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:12.707802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-749b621d-7936-4463-9610-300e4631064c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:12.716985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1269/2531] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:12.721192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.734047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99c87c-8c49-4962-ba72-d51ad8d766d4 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.740102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.740738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.741089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.741502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.743525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:12.743936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.745619 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.759813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1263/2532] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:12.764186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b621d-7936-4463-9610-300e4631064c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.774967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b621d-7936-4463-9610-300e4631064c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.780730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.782053 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.782330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.782609 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.783358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.785830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.824677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1270/2533] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:12.826406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.833320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:12.838770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0595d-d68d-414e-a02e-643c91487bd3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.924276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.924958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.925295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.925582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.926279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.928252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.953047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1264/2534] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:12 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:12.954550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:12.967280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93f8a-9c26-4e7d-b47f-3fe83f4bc4ba admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:12.979330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:12.980173 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:12.980533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:12.980929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:12.981824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:12.983486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:12.995653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1271/2535] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:13.006157 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.017688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbda0e3-01f8-4668-a5a4-bda1dec8d5bf admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.022459 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab010d70-0175-42c4-863b-956190392184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.023738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab010d70-0175-42c4-863b-956190392184 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.024026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab010d70-0175-42c4-863b-956190392184 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.024299 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab010d70-0175-42c4-863b-956190392184 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.025040 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab010d70-0175-42c4-863b-956190392184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.026634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab010d70-0175-42c4-863b-956190392184 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.042159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1265/2536] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:13.043143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.055325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd065fa5-e5a4-4dcb-a169-ca95da9dbef6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.060754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.061853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.062133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.062407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] PATH_INFO: `/v3/projects/00580f9393014da298270d7d2900fb42/users/f48686f930454ab0b6edf24220cb639e/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.078589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=00580f9393014da298270d7d2900fb42, user_id=f48686f930454ab0b6edf24220cb639e, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.081453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.095481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1272/2537] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:13 2022] PUT /identity/v3/projects/00580f9393014da298270d7d2900fb42/users/f48686f930454ab0b6edf24220cb639e/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:13.107122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.107946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.108340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.137252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] MFA Rules not processed for user `f48686f930454ab0b6edf24220cb639e`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:13.138484 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:13.204907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1266/2538] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:13.209414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.270401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b535287-be5b-4dd0-83e7-26f59c7583ee admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.275046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.275757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.276035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.276307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.277001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.278910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.304506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1273/2539] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:13 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:13.309224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.320366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77499ace-7601-4bbf-942d-a5102aa68216 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.384582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.385276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.385569 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.385844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.386550 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.388543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.410378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1267/2540] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:13 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:13.414300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.425253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136850-7868-4c7a-9f04-d068aeb1afa0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.430168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.430825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.431056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.431418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.432102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.433736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.444780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1274/2541] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:13.450275 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.456778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:13.464586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68abfb1-82da-447a-aa99-1139a06002bc admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.470942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.471872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.472155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.472429 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] PATH_INFO: `/v3/projects/69a4ed5a45164fb8b374632cd891b6ed/users/40006e77eaff4abfaeff44bd538dc52a/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.488305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a4ed5a45164fb8b374632cd891b6ed, user_id=40006e77eaff4abfaeff44bd538dc52a, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.491017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.506750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1268/2542] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:13 2022] PUT /identity/v3/projects/69a4ed5a45164fb8b374632cd891b6ed/users/40006e77eaff4abfaeff44bd538dc52a/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:13.511522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.521899 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2e58b-5d5e-4d6d-bbe2-44080181158d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.583328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:13.592065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec623b-ceda-477e-bd14-af4102a0351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.592771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec623b-ceda-477e-bd14-af4102a0351b admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.593055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec623b-ceda-477e-bd14-af4102a0351b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.593385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec623b-ceda-477e-bd14-af4102a0351b admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.594210 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec623b-ceda-477e-bd14-af4102a0351b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.595873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec623b-ceda-477e-bd14-af4102a0351b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.606975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1275/2543] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:13.611246 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.621647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf470856-caf2-4377-8b5a-491a07a86c2c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.626709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.627627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.627911 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.628215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] PATH_INFO: `/v3/projects/69a4ed5a45164fb8b374632cd891b6ed/users/40006e77eaff4abfaeff44bd538dc52a/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.630219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a4ed5a45164fb8b374632cd891b6ed, user_id=40006e77eaff4abfaeff44bd538dc52a, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.632984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.643624 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1269/2544] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:13 2022] PUT /identity/v3/projects/69a4ed5a45164fb8b374632cd891b6ed/users/40006e77eaff4abfaeff44bd538dc52a/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:13.654379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.654759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.655266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.682616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] MFA Rules not processed for user `40006e77eaff4abfaeff44bd538dc52a`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:13.683888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:13.754966 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1276/2545] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:13.766123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:13.767851 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:13.830739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83eaf3-22ad-458f-94dc-4cfa73847af3 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:13.834331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2ff25-3cd4-419c-ba7b-f2ee259458a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:13.834977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2ff25-3cd4-419c-ba7b-f2ee259458a9 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:13.835285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2ff25-3cd4-419c-ba7b-f2ee259458a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:13.835561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2ff25-3cd4-419c-ba7b-f2ee259458a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:13.845778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2ff25-3cd4-419c-ba7b-f2ee259458a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:13.847402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2ff25-3cd4-419c-ba7b-f2ee259458a9 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:13.860421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1270/2546] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:14.026329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:14.113482 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a841254-f490-4569-9716-f6f4e27335ce None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:14.118393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:14.119326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:14.119606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:14.119876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] PATH_INFO: `/v3/users/f48686f930454ab0b6edf24220cb639e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:14.121144 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48686f930454ab0b6edf24220cb639e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:14.123427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:14.185572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48686f930454ab0b6edf24220cb639e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.204250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48686f930454ab0b6edf24220cb639e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.213848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48686f930454ab0b6edf24220cb639e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.219974 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48686f930454ab0b6edf24220cb639e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.221612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48686f930454ab0b6edf24220cb639e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.235797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:14.247999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1277/2547] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:14 2022] DELETE /identity/v3/users/f48686f930454ab0b6edf24220cb639e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:14.253289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:14.254192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:14.254643 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:14.293240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:14.295085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:14.408325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1271/2548] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:14.415528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:14.417033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:14.489841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6e023-8836-4324-a539-f9c1fd2620be admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:14.494852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:14.495704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:14.496052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:14.496436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:14.509538 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:14.511094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:14.521790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1278/2549] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:14.817313 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:14.819089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:14.834250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f46f6-401d-4475-bf5f-ff9e1f895bbd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:14.840720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:14.842707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:14.843380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:14.843958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] PATH_INFO: `/v3/projects/00580f9393014da298270d7d2900fb42` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:14.845663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=00580f9393014da298270d7d2900fb42)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:14.848202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:14.904130 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00580f9393014da298270d7d2900fb42', 'request_id': 'req-f465d4ab-14d4-404b-8098-f99a35df8dac'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.911667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00580f9393014da298270d7d2900fb42', 'request_id': 'req-f465d4ab-14d4-404b-8098-f99a35df8dac'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.919021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00580f9393014da298270d7d2900fb42', 'request_id': 'req-f465d4ab-14d4-404b-8098-f99a35df8dac'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.920860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:14.935592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] The token cache is being invalidate because project 00580f9393014da298270d7d2900fb42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:14.935888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.936572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:14.938223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1272/2550] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:14 2022] DELETE /identity/v3/projects/00580f9393014da298270d7d2900fb42 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:14.942456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:14.953215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee375e-814a-480b-bfd1-4df67f704de1 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:15.016799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:15.017678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.017972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.018270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] PATH_INFO: `/v3/users/40006e77eaff4abfaeff44bd538dc52a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.019518 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40006e77eaff4abfaeff44bd538dc52a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:15.021451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:15.073542 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40006e77eaff4abfaeff44bd538dc52a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.107827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40006e77eaff4abfaeff44bd538dc52a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.118810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40006e77eaff4abfaeff44bd538dc52a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.122343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40006e77eaff4abfaeff44bd538dc52a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.123113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40006e77eaff4abfaeff44bd538dc52a'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.145818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1279/2551] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:14 2022] DELETE /identity/v3/users/40006e77eaff4abfaeff44bd538dc52a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:15.460886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:15.463320 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:15.481998 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d4ab-14d4-404b-8098-f99a35df8dac admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:15.494199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:15.495877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:15.532722 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:15.572109 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:15.573270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.573629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.573976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] PATH_INFO: `/v3/users/ec1e4f6f7b7549b297e9dcc1e923fb3a` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.575378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1e4f6f7b7549b297e9dcc1e923fb3a)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:15.577505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:15.611366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b6d3e-669c-4b5e-96dd-495c52f36c09 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:15.622211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:15.623310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.623607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.625094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] PATH_INFO: `/v3/projects/69a4ed5a45164fb8b374632cd891b6ed` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.626927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a4ed5a45164fb8b374632cd891b6ed)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:15.629154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:15.651823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1e4f6f7b7549b297e9dcc1e923fb3a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.668393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1e4f6f7b7549b297e9dcc1e923fb3a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.677015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1e4f6f7b7549b297e9dcc1e923fb3a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.688877 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1e4f6f7b7549b297e9dcc1e923fb3a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.689506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b7e5b291-979d-4455-8dfc-23b6203c6ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1e4f6f7b7549b297e9dcc1e923fb3a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.696511 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1273/2552] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:15 2022] DELETE /identity/v3/users/ec1e4f6f7b7549b297e9dcc1e923fb3a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:15.700846 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.701215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.701556 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.702109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a4ed5a45164fb8b374632cd891b6ed', 'request_id': 'req-bf549b87-5e90-4838-888f-06fd4318a59f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.709817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a4ed5a45164fb8b374632cd891b6ed', 'request_id': 'req-bf549b87-5e90-4838-888f-06fd4318a59f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.717000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a4ed5a45164fb8b374632cd891b6ed', 'request_id': 'req-bf549b87-5e90-4838-888f-06fd4318a59f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.719663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:15.721176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:15.731945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] The token cache is being invalidate because project 69a4ed5a45164fb8b374632cd891b6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:15.732603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.733693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-bf549b87-5e90-4838-888f-06fd4318a59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:15.736288 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1280/2553] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:15 2022] DELETE /identity/v3/projects/69a4ed5a45164fb8b374632cd891b6ed => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:15.741283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9deaa-3bed-41ec-b04b-d8ca610f6f43 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.741568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9deaa-3bed-41ec-b04b-d8ca610f6f43 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.741843 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9deaa-3bed-41ec-b04b-d8ca610f6f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.765309 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-32f9deaa-3bed-41ec-b04b-d8ca610f6f43 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:15.768083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-32f9deaa-3bed-41ec-b04b-d8ca610f6f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:15.790931 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1274/2554] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:15.803867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:15.807011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:15.824810 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:15.843902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1281/2555] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:15.854337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.854631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.854918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.877329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:15.878706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:15.888324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1282/2556] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:15.889512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9a06e-b2a2-4adb-b755-112354d6d088 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:15.892425 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:15.893467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:15.894131 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.894409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.894684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.908020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1663f-575b-4b98-a8f4-82a3ecb84b8b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:15.908724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:15.910650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:15.915984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:15.916676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.916959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.917236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.917984 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:15.919111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:15.920239 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:15.925644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1275/2557] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:15.935927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1283/2558] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:15.940789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:15.959545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36154f0-0940-4e25-874e-1a8358e6e954 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:15.967199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:15.968047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:15.968337 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:15.968612 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:15.970692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:15.972639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.010508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1276/2559] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:15 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:16.014636 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.029009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d6c41-ce77-467a-9c90-cff5f3bc9664 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.062246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:16.107808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.108564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.108912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.109195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.109907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.112072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.135633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1284/2560] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:16.140372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.152873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2408-f035-46bb-967f-43db4fa23ea7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.157616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.158289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.158574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.158848 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.159621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.161228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.172211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1277/2561] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:16.176385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.187547 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d0a7e-ae2f-408f-ab7e-1b05ebd7b096 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.192323 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.192992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.193291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.193571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.194301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.196665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.207992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1285/2562] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:16.212744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.228831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe6e7a-a7c4-493b-a0c3-ba0ad5fcb266 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.235141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eccdc5-b748-43c1-a131-ba06d2ea05bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.236003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eccdc5-b748-43c1-a131-ba06d2ea05bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.236290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eccdc5-b748-43c1-a131-ba06d2ea05bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.236578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eccdc5-b748-43c1-a131-ba06d2ea05bd admin admin] PATH_INFO: `/v3/projects/dbbcf3dc20e64b239e990328cb660811/users/95edc305ef204cdb88ba19cfeee17451/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.244316 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.246596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:16.260475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eccdc5-b748-43c1-a131-ba06d2ea05bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbbcf3dc20e64b239e990328cb660811, user_id=95edc305ef204cdb88ba19cfeee17451, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.262921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad20fb-e93c-46eb-b61c-01ae2385eb64 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.265045 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eccdc5-b748-43c1-a131-ba06d2ea05bd admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.268051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.269023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.269317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.269596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] PATH_INFO: `/v3/projects/dfc3da30c23e471581b7c3742e5f6284` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.270824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfc3da30c23e471581b7c3742e5f6284)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.272784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.287575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1278/2563] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:16 2022] PUT /identity/v3/projects/dbbcf3dc20e64b239e990328cb660811/users/95edc305ef204cdb88ba19cfeee17451/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:16.299760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.300062 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.300342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.325902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfc3da30c23e471581b7c3742e5f6284', 'request_id': 'req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:16.327214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] MFA Rules not processed for user `95edc305ef204cdb88ba19cfeee17451`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:16.328450 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:16.332300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfc3da30c23e471581b7c3742e5f6284', 'request_id': 'req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:16.338567 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfc3da30c23e471581b7c3742e5f6284', 'request_id': 'req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:16.351146 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] The token cache is being invalidate because project dfc3da30c23e471581b7c3742e5f6284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:16.351436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:16.352627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-6bca47bd-a5f2-4e80-9512-f86d3e42ea87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:16.354852 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1286/2564] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:16 2022] DELETE /identity/v3/projects/dfc3da30c23e471581b7c3742e5f6284 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:16.359315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0aa95e-aacf-4564-aadd-5f2fd4369037 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.359607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0aa95e-aacf-4564-aadd-5f2fd4369037 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.359881 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0aa95e-aacf-4564-aadd-5f2fd4369037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.378462 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3f0aa95e-aacf-4564-aadd-5f2fd4369037 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:16.379736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3f0aa95e-aacf-4564-aadd-5f2fd4369037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:16.398064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1279/2565] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:16.403448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.404840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:16.442415 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1287/2566] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:16.452527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.452814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.453092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.455118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:16.472420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:16.473626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:16.473812 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e797266-81fc-4d82-b74c-ad21cc3558c1 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.479110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.479806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.480093 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.480368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.481099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.482244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1288/2567] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:16.483171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.486445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.496114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc0b43-e494-486d-82d0-af03342debe9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.501266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.501923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.502232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.502490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.503260 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:16.503571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.505217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.510269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1280/2568] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:16.514263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.519432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1289/2569] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:02:16.523510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.533877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc37bd-ea8f-4752-9bb1-419e995a9494 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.544946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:16.582800 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c3ee-b79f-4a45-90c0-098541e80fea admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.588506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.589180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.589465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.589747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.590439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.592541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.596481 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.597138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.597426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.597700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.598385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.600343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.616653 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1281/2570] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:16.620785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.631316 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c30eb1-10fc-4247-a034-2efac70b94a0 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.632292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1290/2571] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:16.636884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.690585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.691287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.691572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.691850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.692609 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.694240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.698015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b575f99-3ab4-4242-8ce5-3d8013661ea6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.702923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.703618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.703903 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.704179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.707745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1282/2572] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:16.707887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.707887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.708779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.719839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e5b62-1cce-4363-b16b-73855d0f7813 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.724678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.725557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.725858 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.726120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] PATH_INFO: `/v3/projects/97012eefb0294ccda41f45a20ea5a7a7/users/ccba11c264814598b2fed9e50d4cae11/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.727995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1291/2573] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:16.731975 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.742040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698c66-0330-4e9b-aa08-118d8ca1ed9d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.742559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97012eefb0294ccda41f45a20ea5a7a7, user_id=ccba11c264814598b2fed9e50d4cae11, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.745716 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.747639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.748301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.748585 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.748993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.749736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.751678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.759572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1283/2574] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:16 2022] PUT /identity/v3/projects/97012eefb0294ccda41f45a20ea5a7a7/users/ccba11c264814598b2fed9e50d4cae11/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:16.764985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1292/2575] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:16.765153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.766003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.777101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5492eb0-2d13-45c4-911f-1c07b3cc68fe admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.800251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:16.829432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a45270-37f9-4f8e-9e5b-0ac9a892428f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.834110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.834783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.835089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.835376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.835476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.836644 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.836766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.837030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.837306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.838032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.838457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.839648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.852272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1284/2576] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:16.852479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1293/2577] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:16.853835 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.854115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:16.864243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ed1c-25b7-4c56-a254-109cc3f48338 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.864434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e30f4-1c02-43cf-a9ef-dbd295be7729 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:16.869060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b1607-ba30-46cf-a000-2ec709493a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.869879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1607-ba30-46cf-a000-2ec709493a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.870167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1607-ba30-46cf-a000-2ec709493a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.870439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04084b95-13c4-4663-8849-a277a3f3a918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:16.870566 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b1607-ba30-46cf-a000-2ec709493a9c admin admin] PATH_INFO: `/v3/projects/68e5aed507694ce39c9208d937bbeb11/users/077335a417a54b6484f89e45116ea8da/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.871504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084b95-13c4-4663-8849-a277a3f3a918 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.871804 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084b95-13c4-4663-8849-a277a3f3a918 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.872080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04084b95-13c4-4663-8849-a277a3f3a918 admin admin] PATH_INFO: `/v3/projects/97012eefb0294ccda41f45a20ea5a7a7/users/ccba11c264814598b2fed9e50d4cae11/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.874110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04084b95-13c4-4663-8849-a277a3f3a918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97012eefb0294ccda41f45a20ea5a7a7, user_id=ccba11c264814598b2fed9e50d4cae11, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.877100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04084b95-13c4-4663-8849-a277a3f3a918 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.882384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:16.887227 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b1607-ba30-46cf-a000-2ec709493a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e5aed507694ce39c9208d937bbeb11, user_id=077335a417a54b6484f89e45116ea8da, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:16.889951 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b1607-ba30-46cf-a000-2ec709493a9c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:16.890106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1285/2578] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:16 2022] PUT /identity/v3/projects/97012eefb0294ccda41f45a20ea5a7a7/users/ccba11c264814598b2fed9e50d4cae11/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:16.902027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.902318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.902598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.907275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1294/2579] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:16 2022] PUT /identity/v3/projects/68e5aed507694ce39c9208d937bbeb11/users/077335a417a54b6484f89e45116ea8da/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:16.920442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:16.920950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:16.921442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:16.933342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] MFA Rules not processed for user `ccba11c264814598b2fed9e50d4cae11`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:16.934586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:16.952541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] MFA Rules not processed for user `077335a417a54b6484f89e45116ea8da`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:16.953937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:17.007146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1286/2580] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:17.019099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:17.020487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:17.021600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1295/2581] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:17.034429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:17.035825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:17.096374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eaccc-fc36-4d19-b4b2-3eb5e8aef668 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:17.101285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:17.101424 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e81ea-f1c8-4f60-9ddc-b38e3c210508 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:17.101983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:17.102269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:17.102639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:17.105656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176585d-9ead-4092-afb6-86a01f9ba1af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:17.106292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176585d-9ead-4092-afb6-86a01f9ba1af service glance] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:17.106656 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176585d-9ead-4092-afb6-86a01f9ba1af service glance] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:17.106940 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176585d-9ead-4092-afb6-86a01f9ba1af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:17.113589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:17.115882 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:17.116373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176585d-9ead-4092-afb6-86a01f9ba1af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:17.120168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176585d-9ead-4092-afb6-86a01f9ba1af service glance] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:17.127861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1287/2582] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:17.131118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1296/2583] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:17.640418 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:17.688745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:17.706503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65cbff8-9842-4b86-aa1f-910086afcb57 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:17.711086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:17.711965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:17.712247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:17.712523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] PATH_INFO: `/v3/users/95edc305ef204cdb88ba19cfeee17451` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:17.713713 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95edc305ef204cdb88ba19cfeee17451)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:17.715708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:17.762048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95edc305ef204cdb88ba19cfeee17451'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:17.774188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95edc305ef204cdb88ba19cfeee17451'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:17.781118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95edc305ef204cdb88ba19cfeee17451'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:17.789508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95edc305ef204cdb88ba19cfeee17451'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:17.790233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95edc305ef204cdb88ba19cfeee17451'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:17.796531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1288/2584] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:17 2022] DELETE /identity/v3/users/95edc305ef204cdb88ba19cfeee17451 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:17.801082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:17.801393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:17.801799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:17.820353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:17.821725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:17.886109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1297/2585] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:17.895855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:17.897262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:17.960464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bce3f-2a30-44b2-9075-806e8a7ba8c6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:17.964516 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:17.965166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:17.965448 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:17.965720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:17.974892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:17.976366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:17.988400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1289/2586] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:18.268778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:18.270297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:18.282375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ae7cf-e05c-479f-8323-a80531526b47 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:18.287100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:18.287952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:18.288232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:18.288863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] PATH_INFO: `/v3/projects/dbbcf3dc20e64b239e990328cb660811` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:18.289795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbbcf3dc20e64b239e990328cb660811)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:18.291778 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:18.316310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:18.343111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbbcf3dc20e64b239e990328cb660811', 'request_id': 'req-0cfd46ca-6844-455f-8876-f2e40311ce9c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.349301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbbcf3dc20e64b239e990328cb660811', 'request_id': 'req-0cfd46ca-6844-455f-8876-f2e40311ce9c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.354969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbbcf3dc20e64b239e990328cb660811', 'request_id': 'req-0cfd46ca-6844-455f-8876-f2e40311ce9c'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.386287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] The token cache is being invalidate because project dbbcf3dc20e64b239e990328cb660811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:18.386973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.391008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.392863 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1298/2587] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:18 2022] DELETE /identity/v3/projects/dbbcf3dc20e64b239e990328cb660811 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:18.397398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:18.413171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ed4c0-68f1-4823-a706-5140bbe33afc service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:18.450818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:18.452438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:18.478251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:18.479174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:18.479463 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:18.479742 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] PATH_INFO: `/v3/users/ccba11c264814598b2fed9e50d4cae11` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:18.481001 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccba11c264814598b2fed9e50d4cae11)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:18.482937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:18.488900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:18.522526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd46ca-6844-455f-8876-f2e40311ce9c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:18.526758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:18.527451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:18.527750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:18.528019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:18.529947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:18.541477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccba11c264814598b2fed9e50d4cae11'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.543342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:18.544844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:18.555328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1299/2588] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:18.555781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccba11c264814598b2fed9e50d4cae11'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.563036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccba11c264814598b2fed9e50d4cae11'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.570853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccba11c264814598b2fed9e50d4cae11'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.571784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b62a4297-e112-447e-ab73-af55a7b0fd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccba11c264814598b2fed9e50d4cae11'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:18.578524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1290/2589] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:18 2022] DELETE /identity/v3/users/ccba11c264814598b2fed9e50d4cae11 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:18.866924 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:18.868750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:18.883088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4452bee-4b34-486d-bb9d-52d898176e7a service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:18.958081 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:18.959197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:18.959478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:18.959750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] PATH_INFO: `/v3/projects/97012eefb0294ccda41f45a20ea5a7a7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:18.961693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=97012eefb0294ccda41f45a20ea5a7a7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:18.963756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.011614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97012eefb0294ccda41f45a20ea5a7a7', 'request_id': 'req-09ccb796-2143-470f-8b10-1f9845f7e8ca'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:19.012712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:19.023980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97012eefb0294ccda41f45a20ea5a7a7', 'request_id': 'req-09ccb796-2143-470f-8b10-1f9845f7e8ca'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:19.031119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97012eefb0294ccda41f45a20ea5a7a7', 'request_id': 'req-09ccb796-2143-470f-8b10-1f9845f7e8ca'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:19.043818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] The token cache is being invalidate because project 97012eefb0294ccda41f45a20ea5a7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:19.044246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:19.044889 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-09ccb796-2143-470f-8b10-1f9845f7e8ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:19.050000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1300/2590] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:18 2022] DELETE /identity/v3/projects/97012eefb0294ccda41f45a20ea5a7a7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:19.051621 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.052085 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.052315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.071504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:19.072823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:19.142339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1291/2591] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:19.151338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.151690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.151926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.171762 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:19.173057 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:19.183453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1301/2592] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:19.187746 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.200504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a4f6f-26f9-49dc-8db1-74266822d1dc None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.208439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.208889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.209163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.209434 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.210181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:19.210537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.212500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.228047 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1292/2593] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:19.229340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.239274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc475bca-0a2d-4e86-bd7a-305b4609659d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.244885 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.245560 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.245869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.246151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.247167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.251408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.281865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1302/2594] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:19.283048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.295372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e100-616b-4162-962d-8a91fe11e319 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.356362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.357035 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.357310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.357593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.358282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.360236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.389618 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1293/2595] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:19.390540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.401831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d70fa2-d263-43e0-a5df-287023b4f11a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.406476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.407172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.407449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.407785 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.408523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.410124 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.421422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1303/2596] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:19.425626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.432172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:19.439387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49712a-6d83-40dc-bd91-f62f0f3eb8ed admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.446476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5aa41-2761-4249-ad42-8a91a05caab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.447199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5aa41-2761-4249-ad42-8a91a05caab2 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.447489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5aa41-2761-4249-ad42-8a91a05caab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.447768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5aa41-2761-4249-ad42-8a91a05caab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.448536 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5aa41-2761-4249-ad42-8a91a05caab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.450326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5aa41-2761-4249-ad42-8a91a05caab2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.460906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1294/2597] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:19.465254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.475422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abee4d2-9bc4-486e-946c-0fa2bdffc010 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.482810 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.483965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.484711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.484711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] PATH_INFO: `/v3/projects/85316886ecd24fc7a70af1f7c5bf0d99/users/42c0c56491aa454a8c2bdd40e047908e/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.501952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85316886ecd24fc7a70af1f7c5bf0d99, user_id=42c0c56491aa454a8c2bdd40e047908e, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.504910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.529265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1304/2598] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:19 2022] PUT /identity/v3/projects/85316886ecd24fc7a70af1f7c5bf0d99/users/42c0c56491aa454a8c2bdd40e047908e/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:19.539592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.539886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.540170 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.570637 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] MFA Rules not processed for user `42c0c56491aa454a8c2bdd40e047908e`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:19.572059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:19.663136 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1295/2599] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:19.668900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.743346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238ebce-bb08-46a8-bfdf-607c158f0783 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.748045 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.748711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.749041 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.749273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.749995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.752204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.784804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1305/2600] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:19.789582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.800884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6f909-553d-4d14-833f-e6a97c2924a3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.864629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.865391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.865747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.866160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.871354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.873413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.911187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1296/2601] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:19 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:19.913107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.925087 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d861fe-9919-4f90-bc09-f03cb8187bb0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.930540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.931135 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.931886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.931886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.932400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.934038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:19.945782 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1306/2602] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:19.949888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:19.963202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a1228-789c-4de1-8553-bfd35a69dc33 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:19.968089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:19.968981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:19.969295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:19.969577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] PATH_INFO: `/v3/projects/6af02572fd8146b99ca60b05e1e0b4f1/users/27f62dc8f7164ad1a553b8016f43b2d8/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:19.986067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af02572fd8146b99ca60b05e1e0b4f1, user_id=27f62dc8f7164ad1a553b8016f43b2d8, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:19.988956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:20.005044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1297/2603] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:19 2022] PUT /identity/v3/projects/6af02572fd8146b99ca60b05e1e0b4f1/users/27f62dc8f7164ad1a553b8016f43b2d8/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:20.009774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:20.020222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55192e7d-0ff6-49f8-9eaa-50de46dffde5 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:20.092400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:20.125008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd2154-6006-40f6-93d1-46306afdab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:20.126548 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd2154-6006-40f6-93d1-46306afdab8f admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:20.126929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd2154-6006-40f6-93d1-46306afdab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:20.127308 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd2154-6006-40f6-93d1-46306afdab8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:20.128272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd2154-6006-40f6-93d1-46306afdab8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:20.129956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd2154-6006-40f6-93d1-46306afdab8f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:20.142059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1307/2604] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:20.146253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:20.165557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecc0b7-a215-47a9-bc2e-6c29788e3e4e admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:20.172719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:20.174052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:20.174334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:20.174605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] PATH_INFO: `/v3/projects/6af02572fd8146b99ca60b05e1e0b4f1/users/27f62dc8f7164ad1a553b8016f43b2d8/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:20.177578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6af02572fd8146b99ca60b05e1e0b4f1, user_id=27f62dc8f7164ad1a553b8016f43b2d8, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:20.182676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:20.207522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1298/2605] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:20 2022] PUT /identity/v3/projects/6af02572fd8146b99ca60b05e1e0b4f1/users/27f62dc8f7164ad1a553b8016f43b2d8/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:20.210220 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:20.210529 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:20.210832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:20.239791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] MFA Rules not processed for user `27f62dc8f7164ad1a553b8016f43b2d8`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:20.242759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:20.336433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1308/2606] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:20.353141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:20.354992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:20.406803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:20.432139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb72b68-27f3-4c53-a80c-cc72c93c05ca admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:20.436807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:20.437462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:20.437805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:20.438007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:20.506641 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:20.509365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:20.520465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1299/2607] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:21.786381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:21.788397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:21.871904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f16ba7-9c9c-40a3-97cb-36dc5f88bc1a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:21.877519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:21.878413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:21.878695 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:21.878965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] PATH_INFO: `/v3/users/fb27ee14314f42cc87e8e6bac2031c70` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:21.880235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb27ee14314f42cc87e8e6bac2031c70)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:21.882197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:21.948917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb27ee14314f42cc87e8e6bac2031c70'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:21.966012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb27ee14314f42cc87e8e6bac2031c70'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:21.978046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:21.978046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb27ee14314f42cc87e8e6bac2031c70'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:21.978046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb27ee14314f42cc87e8e6bac2031c70'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:21.978046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-1f0bb453-05bf-4e31-9da1-49c8d6f740e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb27ee14314f42cc87e8e6bac2031c70'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:21.992558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1309/2608] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:21 2022] DELETE /identity/v3/users/fb27ee14314f42cc87e8e6bac2031c70 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:22.228724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.244059 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5eb1-bd23-43ec-975b-d171a92c46b4 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.309789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.310638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.310917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.311225 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] PATH_INFO: `/v3/projects/37e4a1ceb7b94c4ea11fbbeb8ee0c006` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.312407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e4a1ceb7b94c4ea11fbbeb8ee0c006)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.314287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.365364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e4a1ceb7b94c4ea11fbbeb8ee0c006', 'request_id': 'req-fc302c4a-a01a-48f7-bc64-3f45145118dc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:22.372159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e4a1ceb7b94c4ea11fbbeb8ee0c006', 'request_id': 'req-fc302c4a-a01a-48f7-bc64-3f45145118dc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:22.378506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e4a1ceb7b94c4ea11fbbeb8ee0c006', 'request_id': 'req-fc302c4a-a01a-48f7-bc64-3f45145118dc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:22.392086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] The token cache is being invalidate because project 37e4a1ceb7b94c4ea11fbbeb8ee0c006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:22.392424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:22.393199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-fc302c4a-a01a-48f7-bc64-3f45145118dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:22.395123 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1300/2609] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:22 2022] DELETE /identity/v3/projects/37e4a1ceb7b94c4ea11fbbeb8ee0c006 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:22.402420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.402712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.402989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.424040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:22.425274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:22.486500 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1310/2610] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:22.495300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.495587 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.495866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.515473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:22.516799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:22.526427 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1301/2611] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:22.530685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.542584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21a80a-508f-406e-840c-8022b233aea5 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.547864 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.548672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.549035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.549321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.550069 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:22.550381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.552252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.565675 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1311/2612] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:22.570786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.581127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9f1fc-2712-4fde-9b68-6158ff630b0c None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.585952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.586616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.586894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.587236 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.587929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.589799 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.598149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:22.618533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1302/2613] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:22 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:22.622770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.633142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809390da-2816-4272-ad4a-5ae7a363241e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.689268 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.689965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.690243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.690517 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.691238 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.693143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.712969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1312/2614] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:22 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:22.717097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.727961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24354a-3313-49af-9ca6-fe2988129f2a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.732629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.733287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.733578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.733859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.734588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.736206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.747408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1303/2615] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:22.753265 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.763990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9357da1-b0a9-4765-9073-eaceea40f467 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.768831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fb77d-3747-40ad-a565-49db15b2612b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.769581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fb77d-3747-40ad-a565-49db15b2612b admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.769860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fb77d-3747-40ad-a565-49db15b2612b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.770182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fb77d-3747-40ad-a565-49db15b2612b admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.771029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fb77d-3747-40ad-a565-49db15b2612b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.772729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fb77d-3747-40ad-a565-49db15b2612b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.782565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1313/2616] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:22.786655 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.797271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45d183-8b89-4596-a6c6-5d725b457f5f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:22.801896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:22.802771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.803051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.803356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] PATH_INFO: `/v3/projects/3eb20c3efc384488bbf02eac464e377c/users/eb5a2516965b40b5920afe1124ebb20e/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.818705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb20c3efc384488bbf02eac464e377c, user_id=eb5a2516965b40b5920afe1124ebb20e, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:22.821440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:22.838787 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1304/2617] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:22 2022] PUT /identity/v3/projects/3eb20c3efc384488bbf02eac464e377c/users/eb5a2516965b40b5920afe1124ebb20e/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:22.844044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c15d1-518d-4a73-8cd3-85c965971781 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:22.844588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c15d1-518d-4a73-8cd3-85c965971781 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:22.844588 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c15d1-518d-4a73-8cd3-85c965971781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:22.863151 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ac5c15d1-518d-4a73-8cd3-85c965971781 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:22.864370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ac5c15d1-518d-4a73-8cd3-85c965971781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:22.916345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:22.932890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1314/2618] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:22.939515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:22.941202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:23.000022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b41d4-ddbc-490d-8387-0c63fab52efe admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:23.003924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:23.004567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:23.004847 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:23.005120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:23.018783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:23.018783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:23.027881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1305/2619] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:23.700533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:23.700879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:23.701168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:23.730653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] MFA Rules not processed for user `eb5a2516965b40b5920afe1124ebb20e`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:23.732019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:23.804498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1315/2620] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:23.809110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:23.819220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd7ca3-706f-4cef-bf15-b3303ac77aaa service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:23.824614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:23.825373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:23.825765 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:23.826079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:23.826988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:23.829022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:23.860590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1306/2621] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:23 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:23.864640 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:23.875948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239571d-f16a-4841-b12e-b22c2bec3f80 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:23.943307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:23.944284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:23.944865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:23.944967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:23.945662 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:23.948403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:23.975436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1316/2622] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:23 2022] POST /identity/v3/users => generated 308 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:23.976639 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:23.987013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd1c53-4831-40c1-8ed3-1fa026fb6737 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:23.992056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:23.992853 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:23.993206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:23.993555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:23.994315 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:23.995975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:24.007574 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1307/2623] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:24.012463 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:24.022791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f27f-8c2b-485b-a8ed-e14964a53dc6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:24.028205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:24.029073 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:24.029353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:24.029620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] PATH_INFO: `/v3/projects/4d7dcfedc3e543c89fcd8b3f0e03d45a/users/e0e45d2e028248b083231c8dc5f894a4/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:24.044384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7dcfedc3e543c89fcd8b3f0e03d45a, user_id=e0e45d2e028248b083231c8dc5f894a4, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:24.047161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:24.060689 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1317/2624] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:24 2022] PUT /identity/v3/projects/4d7dcfedc3e543c89fcd8b3f0e03d45a/users/e0e45d2e028248b083231c8dc5f894a4/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:24.064852 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:24.075789 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f30637-0827-437b-bbb7-ef20d717c26d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:24.136689 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfb1d8-b740-4763-9e82-2abb56e8b462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:24.137364 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfb1d8-b740-4763-9e82-2abb56e8b462 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:24.137646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfb1d8-b740-4763-9e82-2abb56e8b462 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:24.137922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfb1d8-b740-4763-9e82-2abb56e8b462 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:24.138649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfb1d8-b740-4763-9e82-2abb56e8b462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:24.140283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfb1d8-b740-4763-9e82-2abb56e8b462 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:24.149996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1308/2625] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:24.154204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:24.164422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bddf8-8648-4005-9949-2633398bd743 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:24.170262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:24.171169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:24.171456 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:24.171732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] PATH_INFO: `/v3/projects/4d7dcfedc3e543c89fcd8b3f0e03d45a/users/e0e45d2e028248b083231c8dc5f894a4/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:24.173770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d7dcfedc3e543c89fcd8b3f0e03d45a, user_id=e0e45d2e028248b083231c8dc5f894a4, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:24.176492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:24.189648 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1318/2626] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:24 2022] PUT /identity/v3/projects/4d7dcfedc3e543c89fcd8b3f0e03d45a/users/e0e45d2e028248b083231c8dc5f894a4/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:24.828248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:24.828784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:24.828784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:24.857188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] MFA Rules not processed for user `e0e45d2e028248b083231c8dc5f894a4`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:24.858518 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:24.871514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:24.942388 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1309/2627] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:26.009707 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:26.054358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:26.079168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d195106-372d-400a-8867-a5f05ec86cdc admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:26.083999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:26.084953 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:26.085372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:26.085666 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] PATH_INFO: `/v3/users/eb5a2516965b40b5920afe1124ebb20e` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:26.087392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5a2516965b40b5920afe1124ebb20e)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:26.089461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:26.142444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5a2516965b40b5920afe1124ebb20e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.154178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5a2516965b40b5920afe1124ebb20e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.162252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5a2516965b40b5920afe1124ebb20e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.163122 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5a2516965b40b5920afe1124ebb20e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.163734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5a2516965b40b5920afe1124ebb20e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.177980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1319/2628] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:26 2022] DELETE /identity/v3/users/eb5a2516965b40b5920afe1124ebb20e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:26.287038 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:26.288607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:26.353964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759c4e0-5215-49c6-9c42-e7cf4e280e0f None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:26.369724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:26.371350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:26.421773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:26.422638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:26.422930 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:26.424153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:26.426205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:26.457460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f8b1f-38aa-47c0-9cb6-b1fb052ef21d admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:26.462995 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:26.463961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:26.464320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:26.464669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] PATH_INFO: `/v3/projects/3eb20c3efc384488bbf02eac464e377c` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:26.465911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb20c3efc384488bbf02eac464e377c)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:26.467905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:26.497111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:26.498942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:26.512760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1310/2629] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:26.523109 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb20c3efc384488bbf02eac464e377c', 'request_id': 'req-273ae687-6933-4e7d-b62f-f2f3b8ae3941'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.530307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb20c3efc384488bbf02eac464e377c', 'request_id': 'req-273ae687-6933-4e7d-b62f-f2f3b8ae3941'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.534234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:26.536561 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb20c3efc384488bbf02eac464e377c', 'request_id': 'req-273ae687-6933-4e7d-b62f-f2f3b8ae3941'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.551495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] The token cache is being invalidate because project 3eb20c3efc384488bbf02eac464e377c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:26.551927 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.552724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.554320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1320/2630] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:26 2022] DELETE /identity/v3/projects/3eb20c3efc384488bbf02eac464e377c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:26.559646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:26.572670 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1e041-0321-4b2a-9100-31e2792e370e service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:26.581460 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:26.642807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:26.643709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:26.643994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:26.644269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] PATH_INFO: `/v3/users/e0e45d2e028248b083231c8dc5f894a4` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:26.645489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e45d2e028248b083231c8dc5f894a4)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:26.647447 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:26.708324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e45d2e028248b083231c8dc5f894a4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.727108 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e45d2e028248b083231c8dc5f894a4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.731973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e45d2e028248b083231c8dc5f894a4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.739623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e45d2e028248b083231c8dc5f894a4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.740286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b0f8d11d-87ee-43ca-aec8-fa6667c13fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e45d2e028248b083231c8dc5f894a4'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:26.747645 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1311/2631] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:26 2022] DELETE /identity/v3/users/e0e45d2e028248b083231c8dc5f894a4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:26.955783 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:26.957290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:26.971490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ae687-6933-4e7d-b62f-f2f3b8ae3941 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.029970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.030824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.031237 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.031531 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] PATH_INFO: `/v3/projects/4d7dcfedc3e543c89fcd8b3f0e03d45a` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.032721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d7dcfedc3e543c89fcd8b3f0e03d45a)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.034626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.080873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d7dcfedc3e543c89fcd8b3f0e03d45a', 'request_id': 'req-8176163c-cef9-4bd7-b704-3730a83c8e59'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:27.088559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d7dcfedc3e543c89fcd8b3f0e03d45a', 'request_id': 'req-8176163c-cef9-4bd7-b704-3730a83c8e59'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:27.094349 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d7dcfedc3e543c89fcd8b3f0e03d45a', 'request_id': 'req-8176163c-cef9-4bd7-b704-3730a83c8e59'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:27.107708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] The token cache is being invalidate because project 4d7dcfedc3e543c89fcd8b3f0e03d45a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:27.108042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:27.108659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8176163c-cef9-4bd7-b704-3730a83c8e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:27.110304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1321/2632] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:26 2022] DELETE /identity/v3/projects/4d7dcfedc3e543c89fcd8b3f0e03d45a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:27.115307 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.115603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.115883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.134919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:27.136211 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:27.207033 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1312/2633] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:27.215761 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.216051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.216358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.236429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:27.237661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:27.246453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1322/2634] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:27.251276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.261594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5cdfe-b548-46a3-8b83-38ff9c8508e2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.266849 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.267561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.267844 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.268118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.268856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:27.269167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.270770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.283440 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1313/2635] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:27.287968 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.297961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f8432-17dd-4550-8236-0ad86eaec207 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.302840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.303530 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.303814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.304115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.304809 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.306933 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.333024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1323/2636] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:27.337703 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.347815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60260409-0d3f-41ad-b7b5-5c0b7ed7ae40 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.360732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:27.408973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.409685 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.409968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.410244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.411073 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.412999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.433814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1314/2637] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:27.437967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.448627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd241db0-9dcc-427f-bb5b-4879f94cccba admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.453404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.454062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.454346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.454651 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.455418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.457080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.468581 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1324/2638] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:27.472676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.483554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c7cb-3db7-4e76-bab2-d1f1284104ee admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.488223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4999907-78ed-404b-8a8c-ad4d9b3a8497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.488887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4999907-78ed-404b-8a8c-ad4d9b3a8497 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.489173 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4999907-78ed-404b-8a8c-ad4d9b3a8497 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.489456 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4999907-78ed-404b-8a8c-ad4d9b3a8497 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.490188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4999907-78ed-404b-8a8c-ad4d9b3a8497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.491820 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4999907-78ed-404b-8a8c-ad4d9b3a8497 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.502410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1315/2639] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:27.506591 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.517082 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c6db9-389c-4c1b-8a74-13bcead5f099 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.521795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.522665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.522948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.523279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] PATH_INFO: `/v3/projects/2e49712d9d6f467188efa375e2aa9a80/users/6834fd3725c54f3987ce3f2ad6675cc7/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.539484 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e49712d9d6f467188efa375e2aa9a80, user_id=6834fd3725c54f3987ce3f2ad6675cc7, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.542682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.556336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1325/2640] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:27 2022] PUT /identity/v3/projects/2e49712d9d6f467188efa375e2aa9a80/users/6834fd3725c54f3987ce3f2ad6675cc7/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:27.567699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.567987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.568285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.596034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] MFA Rules not processed for user `6834fd3725c54f3987ce3f2ad6675cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:27.597188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:27.668402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1316/2641] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:27.672959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.686142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:27.738256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffb7b7-3e1d-4beb-8767-3508ea327fd3 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.743594 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.744255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.744541 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.744817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.745527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.747480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.773348 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1326/2642] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:27.777606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.788635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee7e5f-73c9-4123-b9a5-09d3e59a0587 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.846576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.847198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.847553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.847766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.848467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.850423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.882813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1317/2643] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:27 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:27.886900 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.897386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d896ac-1558-47e0-88b0-2438e9d15246 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.902408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.903107 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.903398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.903676 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.904445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.906314 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.922216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1327/2644] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:27.923471 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.934312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f1417-3975-4a2f-ab10-0694d5030404 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.939160 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfdaf7-2ebf-4098-bc58-215dfc017f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.939975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfdaf7-2ebf-4098-bc58-215dfc017f80 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.940078 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfdaf7-2ebf-4098-bc58-215dfc017f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.940360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfdaf7-2ebf-4098-bc58-215dfc017f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.941125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfdaf7-2ebf-4098-bc58-215dfc017f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.943149 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfdaf7-2ebf-4098-bc58-215dfc017f80 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:27.953754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1318/2645] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:27.958103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:27.969457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b242ef-5de0-407f-be1a-cb3f4b80cec9 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:27.974254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:27.975161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:27.975451 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:27.975748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] PATH_INFO: `/v3/projects/aa4e4a19fd4f4de5b490830694bc5ede/users/f383e2c0afd4426db5bc682735114183/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:27.992104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4e4a19fd4f4de5b490830694bc5ede, user_id=f383e2c0afd4426db5bc682735114183, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:27.995104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.008969 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1328/2646] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:27 2022] PUT /identity/v3/projects/aa4e4a19fd4f4de5b490830694bc5ede/users/f383e2c0afd4426db5bc682735114183/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:28.019487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.019777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.020056 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.027561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:28.050209 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] MFA Rules not processed for user `f383e2c0afd4426db5bc682735114183`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:28.051859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:28.119289 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1319/2647] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:28.124139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.187779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b71e9-0319-4954-baff-66c30c11fbc4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.192557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.193216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.193493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.193858 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.194454 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.196524 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.223688 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1329/2648] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:28 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:28.228214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.239986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82304c23-3974-4e6f-a473-03aca52bda86 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.297295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.297974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.298250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.298526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.299255 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.301348 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.324604 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1320/2649] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:28 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:28.328637 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.339505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbacaf-18d9-4909-b3fb-284149e2062e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.344145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.344795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.345077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.345358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.346086 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.347719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.359806 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1330/2650] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:28.364156 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.374449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4235ea-17a0-43bd-ae21-0a837a7c8eb7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.379191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.380061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.380339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.380610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] PATH_INFO: `/v3/projects/04513d4b72df464b9a4c4f6a88fbae62/users/9d1c825847814b25b4c79e0bcf0652d4/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.395433 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04513d4b72df464b9a4c4f6a88fbae62, user_id=9d1c825847814b25b4c79e0bcf0652d4, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.398052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.417061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1321/2651] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:28 2022] PUT /identity/v3/projects/04513d4b72df464b9a4c4f6a88fbae62/users/9d1c825847814b25b4c79e0bcf0652d4/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:28.421208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.431683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385bafe-5ac5-4a2f-8961-119c04984de0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.466282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:28.490750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebca96-1716-4440-984c-9d8a1e9435ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.491460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebca96-1716-4440-984c-9d8a1e9435ed admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.491749 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebca96-1716-4440-984c-9d8a1e9435ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.492026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebca96-1716-4440-984c-9d8a1e9435ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.492816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebca96-1716-4440-984c-9d8a1e9435ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.494623 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ebca96-1716-4440-984c-9d8a1e9435ed admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.505106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1331/2652] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:28.510262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.520443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c7c16-8ce9-45b0-bdad-5521ae93c454 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.525666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.526528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.526805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.527092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] PATH_INFO: `/v3/projects/04513d4b72df464b9a4c4f6a88fbae62/users/9d1c825847814b25b4c79e0bcf0652d4/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.529023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04513d4b72df464b9a4c4f6a88fbae62, user_id=9d1c825847814b25b4c79e0bcf0652d4, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.532406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.544164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1322/2653] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:28 2022] PUT /identity/v3/projects/04513d4b72df464b9a4c4f6a88fbae62/users/9d1c825847814b25b4c79e0bcf0652d4/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:28.554322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.554620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.554902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.581546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] MFA Rules not processed for user `9d1c825847814b25b4c79e0bcf0652d4`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:28.582743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:28.652017 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1332/2654] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:28.663048 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:28.664949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:28.719982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:28.726412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ad7bf-30f7-4800-975e-8a2a7a70bab4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:28.730341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:28.731017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:28.731334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:28.731613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:28.740733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:28.742188 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:28.753314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1323/2655] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:29.287183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:29.288603 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:29.304211 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a5a9e-d8a1-4d1a-b88b-29cd004eef69 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:29.309147 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:29.309795 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:29.310074 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:29.310347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:29.370857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:29.372442 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:29.381853 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1333/2656] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:34.430981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:34.439768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb206c-76be-473b-b8f4-79b9ac1ef80b service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:34.443540 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329f393-7d38-4b57-8795-1107db7b6f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:34.444227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f393-7d38-4b57-8795-1107db7b6f08 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:34.444492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f393-7d38-4b57-8795-1107db7b6f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:34.444768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329f393-7d38-4b57-8795-1107db7b6f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:34.507298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329f393-7d38-4b57-8795-1107db7b6f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:34.510110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329f393-7d38-4b57-8795-1107db7b6f08 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:34.519759 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1324/2657] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:39.377669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=74 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:39.406315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:39.470767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4baaf4-e25b-4129-9c6c-7a1fa3bec698 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:39.475364 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:39.476289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:39.477236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:39.477515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] PATH_INFO: `/v3/users/6834fd3725c54f3987ce3f2ad6675cc7` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:39.478726 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] RBAC: Authorizing `identity:delete_user(user_id=6834fd3725c54f3987ce3f2ad6675cc7)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:39.480690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:39.527416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6834fd3725c54f3987ce3f2ad6675cc7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:39.540118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6834fd3725c54f3987ce3f2ad6675cc7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:39.547205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6834fd3725c54f3987ce3f2ad6675cc7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:39.547917 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6834fd3725c54f3987ce3f2ad6675cc7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:39.548550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6834fd3725c54f3987ce3f2ad6675cc7'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:39.562444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1334/2658] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:39 2022] DELETE /identity/v3/users/6834fd3725c54f3987ce3f2ad6675cc7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:39.567381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:39.567927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:39.567927 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:39.585394 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:39.587386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:39.649161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1325/2659] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:39.658835 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:39.660296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:39.718665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d0d5e-e6b7-4463-aded-698eb00645be admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:39.722374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:39.723016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:39.723330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:39.723605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:39.733612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:39.735083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:39.747351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1335/2660] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:40.083494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:40.084939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:40.105700 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50009bd4-ca55-4ca7-9df0-2351b5b850f2 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:40.111107 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:40.111957 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:40.112240 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:40.112559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] PATH_INFO: `/v3/projects/2e49712d9d6f467188efa375e2aa9a80` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:40.113705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e49712d9d6f467188efa375e2aa9a80)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:40.115634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:40.162993 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e49712d9d6f467188efa375e2aa9a80', 'request_id': 'req-91e9e3b8-e600-49a1-a18c-72d9993491ba'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.169340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e49712d9d6f467188efa375e2aa9a80', 'request_id': 'req-91e9e3b8-e600-49a1-a18c-72d9993491ba'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.174709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e49712d9d6f467188efa375e2aa9a80', 'request_id': 'req-91e9e3b8-e600-49a1-a18c-72d9993491ba'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.187905 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] The token cache is being invalidate because project 2e49712d9d6f467188efa375e2aa9a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:40.188195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.188817 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.190362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1326/2661] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:40 2022] DELETE /identity/v3/projects/2e49712d9d6f467188efa375e2aa9a80 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:40.194516 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:40.204901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd54f16d-64c3-48aa-b9de-b320ae442370 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:40.222646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:40.262904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:40.263803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:40.264085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:40.264355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] PATH_INFO: `/v3/users/f383e2c0afd4426db5bc682735114183` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:40.265533 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f383e2c0afd4426db5bc682735114183)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:40.267489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:40.315026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f383e2c0afd4426db5bc682735114183'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.328427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f383e2c0afd4426db5bc682735114183'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.335436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f383e2c0afd4426db5bc682735114183'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.336114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f383e2c0afd4426db5bc682735114183'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.336723 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f383e2c0afd4426db5bc682735114183'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.348841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1336/2662] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:40 2022] DELETE /identity/v3/users/f383e2c0afd4426db5bc682735114183 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:40.639167 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:40.641215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:40.657758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9e3b8-e600-49a1-a18c-72d9993491ba admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:40.673438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:40.729975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:40.730841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:40.731159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:40.731439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] PATH_INFO: `/v3/projects/aa4e4a19fd4f4de5b490830694bc5ede` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:40.732699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4e4a19fd4f4de5b490830694bc5ede)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:40.734602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:40.780100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4e4a19fd4f4de5b490830694bc5ede', 'request_id': 'req-7a8990be-bc5b-47f9-82d7-d42f835d7955'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.786057 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4e4a19fd4f4de5b490830694bc5ede', 'request_id': 'req-7a8990be-bc5b-47f9-82d7-d42f835d7955'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.792319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4e4a19fd4f4de5b490830694bc5ede', 'request_id': 'req-7a8990be-bc5b-47f9-82d7-d42f835d7955'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.804118 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] The token cache is being invalidate because project aa4e4a19fd4f4de5b490830694bc5ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:40.804399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.805206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.806753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1327/2663] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:40 2022] DELETE /identity/v3/projects/aa4e4a19fd4f4de5b490830694bc5ede => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:40.810901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:40.812321 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:40.825217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c3590-7f16-4a11-876d-d6a3bb12bcd5 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:40.881339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:40.882195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:40.882471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:40.882781 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] PATH_INFO: `/v3/users/9d1c825847814b25b4c79e0bcf0652d4` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:40.883989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1c825847814b25b4c79e0bcf0652d4)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:40.885920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:40.932306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:40.935690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1c825847814b25b4c79e0bcf0652d4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.949595 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1c825847814b25b4c79e0bcf0652d4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.956653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1c825847814b25b4c79e0bcf0652d4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.957401 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1c825847814b25b4c79e0bcf0652d4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.958002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-19a6e846-43fa-49d3-932f-8a845e3d898d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1c825847814b25b4c79e0bcf0652d4'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:40.973139 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1337/2664] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:40 2022] DELETE /identity/v3/users/9d1c825847814b25b4c79e0bcf0652d4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:41.323855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.336607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:41.358314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8990be-bc5b-47f9-82d7-d42f835d7955 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.430551 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.431512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.431795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.432070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] PATH_INFO: `/v3/projects/04513d4b72df464b9a4c4f6a88fbae62` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.433310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04513d4b72df464b9a4c4f6a88fbae62)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.435356 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.483573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:41.487971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04513d4b72df464b9a4c4f6a88fbae62', 'request_id': 'req-b1e20890-8498-4ed7-a63b-aa2bb759282e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:41.496778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04513d4b72df464b9a4c4f6a88fbae62', 'request_id': 'req-b1e20890-8498-4ed7-a63b-aa2bb759282e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:41.502777 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04513d4b72df464b9a4c4f6a88fbae62', 'request_id': 'req-b1e20890-8498-4ed7-a63b-aa2bb759282e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:41.516104 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] The token cache is being invalidate because project 04513d4b72df464b9a4c4f6a88fbae62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:41.516395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:41.517019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-b1e20890-8498-4ed7-a63b-aa2bb759282e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:41.518585 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1328/2665] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:41 2022] DELETE /identity/v3/projects/04513d4b72df464b9a4c4f6a88fbae62 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:41.524677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.524959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.525232 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.543743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:41.544960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:41.614617 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1338/2666] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:41.624543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.624814 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.625352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.657674 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:41.658895 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:41.668886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1329/2667] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:41.673199 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.684408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6374df-7fc2-4a9d-9247-e0f6776a3e1a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.689101 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.689758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.690042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.690318 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.691046 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:41.691384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.693003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.705744 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1339/2668] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:41.710082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.719894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67297d1-66ef-476f-a69a-c4b7bb7b33c4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.724557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.725200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.725478 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.725753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.726922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.728922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.766991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1330/2669] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:41 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:41.771159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.781544 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1916b-1076-4833-9593-ff48112084c4 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.839313 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.840021 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.840270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.840545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.841284 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.843251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.863335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1340/2670] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:41 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:41.868292 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.879375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d5a93-3f7f-40f1-8d98-15d260e6e900 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.884122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.884782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.885096 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.885396 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.886110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.887744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.897600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1331/2671] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:41.901944 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.913367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2c35e-ce78-433f-9590-6b9560d9069c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.919492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbabf50-b18e-45c2-972d-a39561a14c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.920208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabf50-b18e-45c2-972d-a39561a14c2a admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.920495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabf50-b18e-45c2-972d-a39561a14c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.920775 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabf50-b18e-45c2-972d-a39561a14c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.921583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbabf50-b18e-45c2-972d-a39561a14c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.923243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbabf50-b18e-45c2-972d-a39561a14c2a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.934158 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1341/2672] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:41.938437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:41.948603 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5c276-da58-47a1-9764-c66189a8f3e9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:41.953257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:41.954135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.954417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.954790 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] PATH_INFO: `/v3/projects/b062aaef47c34e08be8751ee05afa921/users/1766033348214a0282651e4720ec6bc7/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:41.969558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b062aaef47c34e08be8751ee05afa921, user_id=1766033348214a0282651e4720ec6bc7, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:41.972248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:41.985806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1332/2673] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:41 2022] PUT /identity/v3/projects/b062aaef47c34e08be8751ee05afa921/users/1766033348214a0282651e4720ec6bc7/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:41.997714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:41.998009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:41.998292 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.024824 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] MFA Rules not processed for user `1766033348214a0282651e4720ec6bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:42.026096 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:42.039774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:42.098155 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1342/2674] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:42.103538 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.166708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19b7b5-1def-4e00-9f7a-94c685b45b29 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.171442 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.172102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.172384 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.172661 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.173354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.175282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.202262 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1333/2675] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:42 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:42.207443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.219048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfd777-36bf-4dfe-be63-c662f1419b45 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.275400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.276070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.276353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.276629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.277326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.279272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.299839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1343/2676] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:42 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:42.304082 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.314576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe5a5c-5e14-4362-8efd-44b700d68460 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.319319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.319980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.320509 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.320984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.321334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.322991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.333663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1334/2677] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:42.339765 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.350948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ece61-c7f2-4da9-8ac7-ff57bef7f279 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.357455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.358329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.358608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.358884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] PATH_INFO: `/v3/projects/129a349ed34b43c0b972c0bc3b370c36/users/7b1525cbf55a434c9add761f9a6f5bd1/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.374061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=129a349ed34b43c0b972c0bc3b370c36, user_id=7b1525cbf55a434c9add761f9a6f5bd1, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.377036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.389929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1344/2678] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:42 2022] PUT /identity/v3/projects/129a349ed34b43c0b972c0bc3b370c36/users/7b1525cbf55a434c9add761f9a6f5bd1/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:42.393953 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.406114 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584fb4a-9513-432a-9bbe-d0d7297f35d2 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.442634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:42.471707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9b6d1-624c-45cc-9196-72b7d9f7b16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.472376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b6d1-624c-45cc-9196-72b7d9f7b16b admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.472656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b6d1-624c-45cc-9196-72b7d9f7b16b admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.472932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9b6d1-624c-45cc-9196-72b7d9f7b16b admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.473657 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b6d1-624c-45cc-9196-72b7d9f7b16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.475281 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9b6d1-624c-45cc-9196-72b7d9f7b16b admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.485878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1335/2679] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:42.490324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.504065 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe582b-2cae-49e1-8653-53647c5c3ccc admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.515154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.516059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.516343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.516619 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] PATH_INFO: `/v3/projects/129a349ed34b43c0b972c0bc3b370c36/users/7b1525cbf55a434c9add761f9a6f5bd1/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.518643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=129a349ed34b43c0b972c0bc3b370c36, user_id=7b1525cbf55a434c9add761f9a6f5bd1, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.521376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.534672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1345/2680] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:42 2022] PUT /identity/v3/projects/129a349ed34b43c0b972c0bc3b370c36/users/7b1525cbf55a434c9add761f9a6f5bd1/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:42.547902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.548214 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.548477 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.575138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] MFA Rules not processed for user `7b1525cbf55a434c9add761f9a6f5bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:42.576361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:42.644026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1336/2681] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:42.655706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:42.657457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:42.716600 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5954e77-bab1-41ec-b07c-d604ed4ddf37 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:42.720120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97b5ad-0141-46af-9a1f-35b951cb90b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:42.720757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97b5ad-0141-46af-9a1f-35b951cb90b0 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:42.721034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97b5ad-0141-46af-9a1f-35b951cb90b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:42.721304 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97b5ad-0141-46af-9a1f-35b951cb90b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:42.730436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97b5ad-0141-46af-9a1f-35b951cb90b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:42.731882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97b5ad-0141-46af-9a1f-35b951cb90b0 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:42.741372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1346/2682] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:43.292541 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:43.354721 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a3b9a-71e5-48b3-991f-9097310a3b7a None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:43.359393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:43.360304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:43.360598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:43.360874 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] PATH_INFO: `/v3/users/1766033348214a0282651e4720ec6bc7` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:43.362080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1766033348214a0282651e4720ec6bc7)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:43.364032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:43.410403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1766033348214a0282651e4720ec6bc7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:43.417244 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:43.422708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1766033348214a0282651e4720ec6bc7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:43.429857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1766033348214a0282651e4720ec6bc7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:43.437099 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1766033348214a0282651e4720ec6bc7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:43.437701 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1766033348214a0282651e4720ec6bc7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:43.444647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1337/2683] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:43 2022] DELETE /identity/v3/users/1766033348214a0282651e4720ec6bc7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:43.451659 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:43.452225 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:43.452645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:43.479811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:43.481047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:43.486161 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:43.543999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1347/2684] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:43.553582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:43.555202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:43.612937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717e52d-40b4-434e-a525-8ae4c9a34fe6 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:43.616598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:43.617451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:43.617805 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:43.618091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:43.628328 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:43.629772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:43.641019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1338/2685] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:43.939420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:43.940960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:43.963647 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec1f5-525e-49a4-8158-fb32d980a1d3 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:43.968578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:43.969452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:43.969731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:43.970001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] PATH_INFO: `/v3/projects/b062aaef47c34e08be8751ee05afa921` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:43.971235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b062aaef47c34e08be8751ee05afa921)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:43.973156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:44.021055 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b062aaef47c34e08be8751ee05afa921', 'request_id': 'req-f057239e-09cb-4fed-8f8d-090ca698be0f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.028060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b062aaef47c34e08be8751ee05afa921', 'request_id': 'req-f057239e-09cb-4fed-8f8d-090ca698be0f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.034110 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b062aaef47c34e08be8751ee05afa921', 'request_id': 'req-f057239e-09cb-4fed-8f8d-090ca698be0f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.048987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] The token cache is being invalidate because project b062aaef47c34e08be8751ee05afa921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:44.049286 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.049913 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.051507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1348/2686] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:43 2022] DELETE /identity/v3/projects/b062aaef47c34e08be8751ee05afa921 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:44.055735 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:44.065829 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380dec3c-6a94-4afd-9f84-9b68b3ee4a31 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:44.123913 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:44.124813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.125362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.125636 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] PATH_INFO: `/v3/users/7b1525cbf55a434c9add761f9a6f5bd1` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.126832 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1525cbf55a434c9add761f9a6f5bd1)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:44.128803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:44.177522 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1525cbf55a434c9add761f9a6f5bd1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.190060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1525cbf55a434c9add761f9a6f5bd1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.198589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1525cbf55a434c9add761f9a6f5bd1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.202278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:44.206602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1525cbf55a434c9add761f9a6f5bd1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.207672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-14e4189a-40da-46d6-b573-2e5d2250c3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1525cbf55a434c9add761f9a6f5bd1'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.215766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1339/2687] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:44 2022] DELETE /identity/v3/users/7b1525cbf55a434c9add761f9a6f5bd1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:44.513150 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:44.515020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:44.531735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057239e-09cb-4fed-8f8d-090ca698be0f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:44.593669 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:44.594682 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.594963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.595266 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] PATH_INFO: `/v3/projects/129a349ed34b43c0b972c0bc3b370c36` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.596461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129a349ed34b43c0b972c0bc3b370c36)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:44.598449 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:44.643331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:44.654471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129a349ed34b43c0b972c0bc3b370c36', 'request_id': 'req-99e20266-659c-4210-a744-4c9d0024d563'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.660820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129a349ed34b43c0b972c0bc3b370c36', 'request_id': 'req-99e20266-659c-4210-a744-4c9d0024d563'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.667093 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129a349ed34b43c0b972c0bc3b370c36', 'request_id': 'req-99e20266-659c-4210-a744-4c9d0024d563'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.681108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] The token cache is being invalidate because project 129a349ed34b43c0b972c0bc3b370c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:44.681400 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.682091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-99e20266-659c-4210-a744-4c9d0024d563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:44.683665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1349/2688] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:44 2022] DELETE /identity/v3/projects/129a349ed34b43c0b972c0bc3b370c36 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:44.689000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.689673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.689673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.725922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:44.727371 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:44.790806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1340/2689] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:44.799019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.799267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.799549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.827009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:44.828287 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:44.836502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1350/2690] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:44.840955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:44.851036 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12982f-719d-41cf-83c9-fee7ced070c1 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:44.856294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:44.856948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.857223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.857495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.858221 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:44.858523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:44.860141 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:44.872420 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1341/2691] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:02:44.876836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:44.886111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f97b5a-957f-4fae-bb03-58d478e8932d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:44.890941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:44.891611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.891897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.892174 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.892876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:44.894776 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:44.921351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1351/2692] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:44 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:44.925588 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:44.936274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a361f-41a5-4ae7-b1ef-8369bac7dc02 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:44.991711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:44.993002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:44.993279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:44.993553 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:44.994267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:44.996187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:45.019512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1342/2693] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:44 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:45.020644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:45.031366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a657ba-d633-41ce-810b-ea3ac437fd06 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:45.036028 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:45.036686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:45.036970 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:45.037250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:45.037979 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:45.040143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:45.050573 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1352/2694] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:45.055017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:45.065502 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa88401-bd52-4315-8c59-4e53991b052c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:45.070898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e53d24-e3b3-46a5-9736-c0f706fbe01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:45.071592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e53d24-e3b3-46a5-9736-c0f706fbe01d admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:45.071879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e53d24-e3b3-46a5-9736-c0f706fbe01d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:45.072158 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e53d24-e3b3-46a5-9736-c0f706fbe01d admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:45.072890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e53d24-e3b3-46a5-9736-c0f706fbe01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:45.074485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e53d24-e3b3-46a5-9736-c0f706fbe01d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:45.085053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1343/2695] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:45.089121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:45.099167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f368e3-7df7-4301-9ba5-689f24bd16fc admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:45.105145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:45.106047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:45.106326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:45.106599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] PATH_INFO: `/v3/projects/40f8d45a1208441eb16991176526d242/users/fd68b7e317d1417e81eefba759004d99/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:45.122067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40f8d45a1208441eb16991176526d242, user_id=fd68b7e317d1417e81eefba759004d99, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:45.124714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:45.138063 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1353/2696] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:45 2022] PUT /identity/v3/projects/40f8d45a1208441eb16991176526d242/users/fd68b7e317d1417e81eefba759004d99/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:45.148617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:45.148914 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:45.149199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:45.175783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] MFA Rules not processed for user `fd68b7e317d1417e81eefba759004d99`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:45.177151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:45.224956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:45.243028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1344/2697] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:45.254379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:45.256121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:45.311763 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048aca8-1dc1-4b2b-960c-2f2f189afe61 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:45.315322 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:45.315957 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:45.316234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:45.316505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:45.326990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:45.328469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:45.338663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1354/2698] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:55.239523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:55.333478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:56.626314 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:56.627785 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:56.650912 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:56.652410 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:56.703183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff67cc7-8810-46f7-a93f-71b3b1c11225 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:56.708300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:56.709217 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:56.709504 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:56.709781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] PATH_INFO: `/v3/users/42c0c56491aa454a8c2bdd40e047908e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:56.711501 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c0c56491aa454a8c2bdd40e047908e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:56.713457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:56.727306 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb93e12-ec24-4df1-86cb-c7baba4c49bf service glance] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:56.731952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:56.732823 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:56.733106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:56.733382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] PATH_INFO: `/v3/users/077335a417a54b6484f89e45116ea8da` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:56.734613 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077335a417a54b6484f89e45116ea8da)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:56.736572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:56.763199 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c0c56491aa454a8c2bdd40e047908e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.778265 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c0c56491aa454a8c2bdd40e047908e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.786586 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c0c56491aa454a8c2bdd40e047908e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.794496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c0c56491aa454a8c2bdd40e047908e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.795115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077335a417a54b6484f89e45116ea8da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.795353 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3dfe7ee8-e3b9-4252-baed-cb8d870f1959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c0c56491aa454a8c2bdd40e047908e'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.801797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1345/2699] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:56 2022] DELETE /identity/v3/users/42c0c56491aa454a8c2bdd40e047908e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:56.807111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:56.807514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:56.807798 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:56.808844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077335a417a54b6484f89e45116ea8da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.815832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077335a417a54b6484f89e45116ea8da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.816519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077335a417a54b6484f89e45116ea8da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.817164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-de3d8185-6362-49e4-be44-8616863c1282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077335a417a54b6484f89e45116ea8da'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:56.825978 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:56.827288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:56.831044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1355/2700] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:56 2022] DELETE /identity/v3/users/077335a417a54b6484f89e45116ea8da => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:56.836217 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:56.836568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:56.837185 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:56.855622 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:56.856754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:56.898464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1346/2701] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:56.909190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:56.911208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:56.923956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1356/2702] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:56.933989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:56.935359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:56.935508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:56.957948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:56.979280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a963caf-fa19-4e96-8db7-f4c2d4eb1c22 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:56.999755 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102aab7-cde4-4d2d-b89f-a099956526b1 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.003730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.004380 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.004665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.004943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.014841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.016343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.027906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1357/2703] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:57.032869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.033638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.033926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.034204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.045605 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.047148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.057584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1347/2704] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:02:57.420070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:57.421520 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:57.437466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd58bca-3b79-482b-b39e-e2fc6b86b101 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.442558 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.443457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.443746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.444033 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] PATH_INFO: `/v3/projects/68e5aed507694ce39c9208d937bbeb11` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.445945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e5aed507694ce39c9208d937bbeb11)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.447988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.461582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:57.463467 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:57.478018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2941402-1ffe-4ae2-b6e5-43c5d6b46f61 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.485226 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.486325 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.486770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.487163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] PATH_INFO: `/v3/projects/85316886ecd24fc7a70af1f7c5bf0d99` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.488526 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85316886ecd24fc7a70af1f7c5bf0d99)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.490638 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.509672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e5aed507694ce39c9208d937bbeb11', 'request_id': 'req-4064592d-7f24-41a5-9be5-fb532a40a5f9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.516138 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e5aed507694ce39c9208d937bbeb11', 'request_id': 'req-4064592d-7f24-41a5-9be5-fb532a40a5f9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.522305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e5aed507694ce39c9208d937bbeb11', 'request_id': 'req-4064592d-7f24-41a5-9be5-fb532a40a5f9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.537059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] The token cache is being invalidate because project 68e5aed507694ce39c9208d937bbeb11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:57.537430 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.538213 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4064592d-7f24-41a5-9be5-fb532a40a5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.540766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1358/2705] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:57 2022] DELETE /identity/v3/projects/68e5aed507694ce39c9208d937bbeb11 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:57.544902 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85316886ecd24fc7a70af1f7c5bf0d99', 'request_id': 'req-65bbdabc-4a68-4697-8fe0-6ea725e42159'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.547106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2ad6-70c5-4b65-9677-51cf7ef15a9b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.547559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2ad6-70c5-4b65-9677-51cf7ef15a9b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.547993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2ad6-70c5-4b65-9677-51cf7ef15a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.552942 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85316886ecd24fc7a70af1f7c5bf0d99', 'request_id': 'req-65bbdabc-4a68-4697-8fe0-6ea725e42159'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.559287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85316886ecd24fc7a70af1f7c5bf0d99', 'request_id': 'req-65bbdabc-4a68-4697-8fe0-6ea725e42159'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.568330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-084c2ad6-70c5-4b65-9677-51cf7ef15a9b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:57.570376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-084c2ad6-70c5-4b65-9677-51cf7ef15a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:57.572581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] The token cache is being invalidate because project 85316886ecd24fc7a70af1f7c5bf0d99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:57.572875 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.573496 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.575043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1348/2706] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:57 2022] DELETE /identity/v3/projects/85316886ecd24fc7a70af1f7c5bf0d99 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:57.578968 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:57.580302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:57.633579 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1359/2707] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:57.641367 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.641729 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.642085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.654915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbdabc-4a68-4697-8fe0-6ea725e42159 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.660649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.661002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:57.661530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.661808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.662081 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] PATH_INFO: `/v3/users/27f62dc8f7164ad1a553b8016f43b2d8` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.662209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:57.663876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27f62dc8f7164ad1a553b8016f43b2d8)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.666168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.672868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:57.676133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1360/2708] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:57.681032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:57.691794 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005acff-815f-4d13-911b-82d712d859f9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.696612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.697275 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.697559 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.698291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.698568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:57.698887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.700668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.713335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1361/2709] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:02:57.717311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:57.725565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27f62dc8f7164ad1a553b8016f43b2d8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.727681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82df0f9-48cf-4f16-b2de-bc0bf62ef622 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.733317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.734319 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.734742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.734874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.735606 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.737565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.745488 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27f62dc8f7164ad1a553b8016f43b2d8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.753087 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27f62dc8f7164ad1a553b8016f43b2d8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.760660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27f62dc8f7164ad1a553b8016f43b2d8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.761261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27f62dc8f7164ad1a553b8016f43b2d8'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:57.768153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1349/2710] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:02:57 2022] DELETE /identity/v3/users/27f62dc8f7164ad1a553b8016f43b2d8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:57.771141 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1362/2711] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:57 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:57.775561 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:57.777021 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:57.879028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3fb0e-1943-4685-9e16-028710230026 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:57.961936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:57.962705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:57.962988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:57.967102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:57.967102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:57.967102 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:57.992750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1350/2712] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:57 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:57.996461 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.008423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a6820-bd3e-406e-8323-8660008b5972 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.013060 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:58.015426 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.015808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.016441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.016704 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.017466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.019483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.039610 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1363/2713] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:58.041317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.058025 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08738fc3-fc45-4e0a-9c93-f39f7195fbfe admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.063326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.064000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.064278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.064548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.065286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.067012 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.083365 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1351/2714] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:02:58.084643 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.102817 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4053d9-e095-445f-ad8b-46b93f90937f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.109169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71da5be6-5e6f-4bab-b563-8656a6fa8bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.110568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da5be6-5e6f-4bab-b563-8656a6fa8bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.111019 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da5be6-5e6f-4bab-b563-8656a6fa8bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.111671 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71da5be6-5e6f-4bab-b563-8656a6fa8bc3 admin admin] PATH_INFO: `/v3/projects/f4a69d59d62145a5bda8136a0e3448c6/users/6acd7109811a44b780cf61a5b707eebc/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.114916 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.116955 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:58.132902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da5be6-5e6f-4bab-b563-8656a6fa8bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a69d59d62145a5bda8136a0e3448c6, user_id=6acd7109811a44b780cf61a5b707eebc, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.135660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf47549-22f1-4c56-a810-c8208a3833a8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.135832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71da5be6-5e6f-4bab-b563-8656a6fa8bc3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.151094 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.151989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.152273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.152545 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] PATH_INFO: `/v3/projects/6af02572fd8146b99ca60b05e1e0b4f1` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.153757 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6af02572fd8146b99ca60b05e1e0b4f1)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.155699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.156855 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1364/2715] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:58 2022] PUT /identity/v3/projects/f4a69d59d62145a5bda8136a0e3448c6/users/6acd7109811a44b780cf61a5b707eebc/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:58.164917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:58.225019 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6af02572fd8146b99ca60b05e1e0b4f1', 'request_id': 'req-ef76d152-da59-43f1-9235-040804a8aadf'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:58.236444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.236983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.236983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.239917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6af02572fd8146b99ca60b05e1e0b4f1', 'request_id': 'req-ef76d152-da59-43f1-9235-040804a8aadf'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:58.251887 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6af02572fd8146b99ca60b05e1e0b4f1', 'request_id': 'req-ef76d152-da59-43f1-9235-040804a8aadf'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:58.266457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] The token cache is being invalidate because project 6af02572fd8146b99ca60b05e1e0b4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:02:58.266457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:58.266457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-ef76d152-da59-43f1-9235-040804a8aadf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:02:58.267989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1352/2716] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:02:58 2022] DELETE /identity/v3/projects/6af02572fd8146b99ca60b05e1e0b4f1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:58.273415 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1398a-0fc3-4f6f-93ca-31c28392db4b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.273707 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1398a-0fc3-4f6f-93ca-31c28392db4b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.273983 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1398a-0fc3-4f6f-93ca-31c28392db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.275631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] MFA Rules not processed for user `6acd7109811a44b780cf61a5b707eebc`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:58.277546 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:58.295722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-fff1398a-0fc3-4f6f-93ca-31c28392db4b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:58.297145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-fff1398a-0fc3-4f6f-93ca-31c28392db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:58.361760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1365/2717] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:58.369626 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1353/2718] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:58.375335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.376879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:58.378945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.379462 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.379842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.404111 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:58.405403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:58.414310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1354/2719] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:58.418593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.433052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a874008-4034-4ace-b99f-43ec06c661f8 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.436340 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.437184 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.437572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.437954 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.438859 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:02:58.439257 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.441027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.456391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1355/2720] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:02:58.462206 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.462404 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937d27e-7fbd-4963-8cdb-d6e4c939deb2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.467325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.468061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.468413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.468685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.473523 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66131e-ee33-4990-8f0c-8d16d31ab07f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.479386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.479828 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.479985 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.480271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.480547 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.481248 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.481770 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.483252 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.494274 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1366/2721] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:02:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:02:58.520084 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1356/2722] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:02:58.524455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.541354 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de095ce-02d6-4585-8434-cad45ca3e3fd service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.620420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.621259 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.621537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.621811 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.622562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.624616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.634165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:58.653077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1367/2723] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:58.657190 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.669185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472bd54-40dd-41e1-9a02-b89379e0b132 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.678414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.679179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.679470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.679743 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.680568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.682175 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.695363 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1357/2724] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:58.700756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.713767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2a25f-b56b-436f-a7ec-51871e672624 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.719698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eae59-ec3e-4b1c-9515-b11401ab7da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.720506 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eae59-ec3e-4b1c-9515-b11401ab7da3 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.720826 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eae59-ec3e-4b1c-9515-b11401ab7da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.721099 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eae59-ec3e-4b1c-9515-b11401ab7da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.722068 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eae59-ec3e-4b1c-9515-b11401ab7da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.727355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eae59-ec3e-4b1c-9515-b11401ab7da3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.744799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1368/2725] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:58.745303 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.757738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45916a0f-0882-4053-966f-b956977b2ee8 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:58.764105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:58.765112 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.765395 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.765747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] PATH_INFO: `/v3/projects/57657c0328db496abeb4cc37d0f88718/users/c41d49271f23488e8a0b1dc9b8f6ffd1/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.784419 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57657c0328db496abeb4cc37d0f88718, user_id=c41d49271f23488e8a0b1dc9b8f6ffd1, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:58.787555 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:58.809696 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1358/2726] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:58 2022] PUT /identity/v3/projects/57657c0328db496abeb4cc37d0f88718/users/c41d49271f23488e8a0b1dc9b8f6ffd1/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:58.820317 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:58.820601 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:58.820697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:58.848433 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] MFA Rules not processed for user `c41d49271f23488e8a0b1dc9b8f6ffd1`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:58.849681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:58.934143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1369/2727] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:02:58.938793 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:58.952996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:59.012974 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b69abf-3d03-4c5c-b8fe-2f99795b111c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.017692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.018382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.018662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.018937 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.019683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.021564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.048424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1359/2728] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:58 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:59.052550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.063583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2422-62b9-4ec0-b1e3-3159ba65f188 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.121990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.122661 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.122937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.123250 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.123943 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.126106 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.149572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1370/2729] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:59 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:59.153771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.164410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137a5b-2e91-4619-a9f9-5b02668bc359 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.170030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.170697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.170987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.171305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.172060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.173691 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.185076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1360/2730] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:59.189270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.201405 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e01068-4364-4dc5-a0f2-126df35d9c02 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.206204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12195b-b67c-4fc5-8332-9e1214c2b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.206907 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12195b-b67c-4fc5-8332-9e1214c2b3fd admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.207222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12195b-b67c-4fc5-8332-9e1214c2b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.207493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12195b-b67c-4fc5-8332-9e1214c2b3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.208221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12195b-b67c-4fc5-8332-9e1214c2b3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.209836 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12195b-b67c-4fc5-8332-9e1214c2b3fd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.221891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1371/2731] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:59.226841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.236963 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e4b0f-22e8-4c63-be4c-b7f871acb592 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.242004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.242889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.243202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.243472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] PATH_INFO: `/v3/projects/77f9822497a141cb9d6028bf08a4a161/users/af415a4973064ca4aeb382f96e8c5705/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.259254 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f9822497a141cb9d6028bf08a4a161, user_id=af415a4973064ca4aeb382f96e8c5705, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.261886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.276302 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1361/2732] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:59 2022] PUT /identity/v3/projects/77f9822497a141cb9d6028bf08a4a161/users/af415a4973064ca4aeb382f96e8c5705/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:59.343919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.344492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.344774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.372296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] MFA Rules not processed for user `af415a4973064ca4aeb382f96e8c5705`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:59.373502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:59.416159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:59.444070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1372/2733] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:02:59.449041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.516991 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b5bd3-3013-4666-81bf-e76b1d7c8297 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.521796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.522494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.523534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.523824 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.524542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.526503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.552451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1362/2734] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:02:59 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:59.557263 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.568281 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9ae5-654a-4768-8c4b-867cc813f57a None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.628061 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.628742 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.629026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.629299 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.629991 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.632136 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.652143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1373/2735] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:02:59 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:02:59.656270 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.669208 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cc9cc-eca1-493d-940c-c262d72b6a1c admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.673992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.674663 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.674966 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.675267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.676152 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.677749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.689023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1363/2736] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:59.693290 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.704094 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a080ad-4b3c-4090-8a31-cb72be29b3fd admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.709047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.709942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.710769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.711079 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] PATH_INFO: `/v3/projects/ea2a8ce87d984758a62e1b1ff2909e58/users/4687bd5bffd84200837a6fa728a92734/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.728993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea2a8ce87d984758a62e1b1ff2909e58, user_id=4687bd5bffd84200837a6fa728a92734, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.731721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.745572 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1374/2737] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:59 2022] PUT /identity/v3/projects/ea2a8ce87d984758a62e1b1ff2909e58/users/4687bd5bffd84200837a6fa728a92734/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:02:59.749598 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.760627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab27f4a-f1c3-4e83-9e3f-1c0e3f331828 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.793326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:02:59.818517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813952c-e770-4c4b-b3b3-c0734a9984a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.819230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813952c-e770-4c4b-b3b3-c0734a9984a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.820294 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813952c-e770-4c4b-b3b3-c0734a9984a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.820924 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813952c-e770-4c4b-b3b3-c0734a9984a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.821673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813952c-e770-4c4b-b3b3-c0734a9984a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.823310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813952c-e770-4c4b-b3b3-c0734a9984a5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.833739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1364/2738] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:02:59.837754 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:02:59.848172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44970851-4282-40ca-bce4-ede154664698 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:02:59.853874 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:02:59.854748 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.855032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.855339 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] PATH_INFO: `/v3/projects/ea2a8ce87d984758a62e1b1ff2909e58/users/4687bd5bffd84200837a6fa728a92734/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.857301 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea2a8ce87d984758a62e1b1ff2909e58, user_id=4687bd5bffd84200837a6fa728a92734, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:02:59.859947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:02:59.871176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1375/2739] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:02:59 2022] PUT /identity/v3/projects/ea2a8ce87d984758a62e1b1ff2909e58/users/4687bd5bffd84200837a6fa728a92734/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:02:59.882155 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:02:59.882472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:02:59.882816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:02:59.910831 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] MFA Rules not processed for user `4687bd5bffd84200837a6fa728a92734`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:02:59.912105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:02:59.996349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1365/2740] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:03:00.008910 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:00.097678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448565b-1117-442f-93ce-4bbea0b5e9a9 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:00.102592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:00.103418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:00.103736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:00.104166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:00.118373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:00.119856 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:00.130334 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1376/2741] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:03:00.171443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:00.181381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325965-564d-48c2-a862-109ddbf3bff4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:00.185061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647eb2d7-9770-4e83-ba42-052208d2a3b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:00.185712 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb2d7-9770-4e83-ba42-052208d2a3b8 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:00.186014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb2d7-9770-4e83-ba42-052208d2a3b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:00.186288 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647eb2d7-9770-4e83-ba42-052208d2a3b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:00.248642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb2d7-9770-4e83-ba42-052208d2a3b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:00.250341 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647eb2d7-9770-4e83-ba42-052208d2a3b8 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:00.259907 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1366/2742] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:03.171715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:03.208735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:03.239353 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c097fa4-753b-4720-a4cd-d27fdc62b0af service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:03.245192 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:03.246059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:03.246338 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:03.246614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] PATH_INFO: `/v3/users/c41d49271f23488e8a0b1dc9b8f6ffd1` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:03.247919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41d49271f23488e8a0b1dc9b8f6ffd1)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:03.249834 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:03.298877 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41d49271f23488e8a0b1dc9b8f6ffd1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.311255 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41d49271f23488e8a0b1dc9b8f6ffd1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.317918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41d49271f23488e8a0b1dc9b8f6ffd1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.318577 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41d49271f23488e8a0b1dc9b8f6ffd1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.319906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41d49271f23488e8a0b1dc9b8f6ffd1'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.332721 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1377/2743] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:03 2022] DELETE /identity/v3/users/c41d49271f23488e8a0b1dc9b8f6ffd1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:03.337807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:03.338305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:03.338406 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:03.356525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:03.357738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:03.408380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:03.419015 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1367/2744] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:03.428679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:03.430310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:03.488356 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683953-c3c6-4525-99ed-f08995325a08 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:03.492296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:03.492941 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:03.493226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:03.493501 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:03.502786 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:03.504361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:03.514251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1378/2745] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:03.797110 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:03.798890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:03.813866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc553ce-cf07-41a1-9b95-1ef8dfe3d7ef None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:03.822548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:03.823666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:03.824125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:03.824492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] PATH_INFO: `/v3/projects/57657c0328db496abeb4cc37d0f88718` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:03.826507 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57657c0328db496abeb4cc37d0f88718)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:03.828662 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:03.912387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57657c0328db496abeb4cc37d0f88718', 'request_id': 'req-f61216dd-abd3-4d0e-b25d-2e98a8727378'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.919472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57657c0328db496abeb4cc37d0f88718', 'request_id': 'req-f61216dd-abd3-4d0e-b25d-2e98a8727378'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.927276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57657c0328db496abeb4cc37d0f88718', 'request_id': 'req-f61216dd-abd3-4d0e-b25d-2e98a8727378'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.941405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] The token cache is being invalidate because project 57657c0328db496abeb4cc37d0f88718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:03.941724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.942370 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:03.944032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1368/2746] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:03 2022] DELETE /identity/v3/projects/57657c0328db496abeb4cc37d0f88718 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:03.948743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:03.958787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de412a-b4ee-4ee2-b170-c0dd3d498e7f service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:04.017062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:04.025235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:04.026386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:04.026790 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:04.027485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] PATH_INFO: `/v3/users/af415a4973064ca4aeb382f96e8c5705` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:04.028772 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af415a4973064ca4aeb382f96e8c5705)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:04.031127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:04.111159 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af415a4973064ca4aeb382f96e8c5705'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.129176 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af415a4973064ca4aeb382f96e8c5705'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.137992 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af415a4973064ca4aeb382f96e8c5705'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.138923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af415a4973064ca4aeb382f96e8c5705'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.147972 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af415a4973064ca4aeb382f96e8c5705'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.162383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1379/2747] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:03 2022] DELETE /identity/v3/users/af415a4973064ca4aeb382f96e8c5705 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:04.435319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:04.436755 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:04.450425 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61216dd-abd3-4d0e-b25d-2e98a8727378 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:04.507600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:04.508464 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:04.508744 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:04.509022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] PATH_INFO: `/v3/projects/77f9822497a141cb9d6028bf08a4a161` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:04.510218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f9822497a141cb9d6028bf08a4a161)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:04.512165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:04.559778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f9822497a141cb9d6028bf08a4a161', 'request_id': 'req-1d2ab325-76f5-45be-8e73-816626d126a7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.566276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f9822497a141cb9d6028bf08a4a161', 'request_id': 'req-1d2ab325-76f5-45be-8e73-816626d126a7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.572375 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f9822497a141cb9d6028bf08a4a161', 'request_id': 'req-1d2ab325-76f5-45be-8e73-816626d126a7'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.586409 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] The token cache is being invalidate because project 77f9822497a141cb9d6028bf08a4a161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:04.586692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.587369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.588945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1369/2748] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:04 2022] DELETE /identity/v3/projects/77f9822497a141cb9d6028bf08a4a161 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:04.593015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:04.594477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:04.607825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c14b7c-caa6-42ef-9356-d43e57f14654 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:04.668961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:04.669830 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:04.670111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:04.670385 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] PATH_INFO: `/v3/users/4687bd5bffd84200837a6fa728a92734` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:04.671629 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4687bd5bffd84200837a6fa728a92734)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:04.673565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:04.721452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4687bd5bffd84200837a6fa728a92734'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.734029 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4687bd5bffd84200837a6fa728a92734'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.740766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4687bd5bffd84200837a6fa728a92734'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.741483 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4687bd5bffd84200837a6fa728a92734'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.742108 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ec0a27ca-345b-4bc8-a5af-0a5c5cd1999d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4687bd5bffd84200837a6fa728a92734'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:04.744844 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:04.758626 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1380/2749] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:04 2022] DELETE /identity/v3/users/4687bd5bffd84200837a6fa728a92734 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:05.037994 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.040135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:05.054558 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ab325-76f5-45be-8e73-816626d126a7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.062808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:05.116929 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.117807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.118079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.118358 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] PATH_INFO: `/v3/projects/ea2a8ce87d984758a62e1b1ff2909e58` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.119614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea2a8ce87d984758a62e1b1ff2909e58)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.121575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.169475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea2a8ce87d984758a62e1b1ff2909e58', 'request_id': 'req-3b2b200a-0508-4d00-b081-cd1c51af93bc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:05.175727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea2a8ce87d984758a62e1b1ff2909e58', 'request_id': 'req-3b2b200a-0508-4d00-b081-cd1c51af93bc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:05.182154 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea2a8ce87d984758a62e1b1ff2909e58', 'request_id': 'req-3b2b200a-0508-4d00-b081-cd1c51af93bc'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:05.198204 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] The token cache is being invalidate because project ea2a8ce87d984758a62e1b1ff2909e58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:05.198494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:05.199168 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-3b2b200a-0508-4d00-b081-cd1c51af93bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:05.200722 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1370/2750] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:05 2022] DELETE /identity/v3/projects/ea2a8ce87d984758a62e1b1ff2909e58 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:05.206750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.206949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.207256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.225846 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:05.227165 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:05.288774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1381/2751] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:05.296867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.297153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.297441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.318410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:05.320361 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:05.329203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1371/2752] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:05.333330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.344039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5848b-3682-45b5-9b21-3418560a3c41 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.348705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.349368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.349646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.349919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.351178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:03:05.351477 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.353072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.364700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1382/2753] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:03:05.369631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.379530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19bd9a0-f457-47d8-8801-d6a08233a465 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.385232 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.385891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.386171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.386451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.387179 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.389080 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.415368 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1372/2754] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:05.419621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.429862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ca140-414d-4175-8a15-60498968db2c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.494508 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.495438 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.495819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.496198 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.497182 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.499423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.521382 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1383/2755] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:05.526241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.536600 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970cb4d-7a36-4aa3-9893-5ab399ebb627 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.542407 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.543222 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.543513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.543797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.544539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.546137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.680117 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1373/2756] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:05.684262 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.694694 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3f0ac-7d67-4974-915c-c4850ae16e8c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.699460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57f3ea-be78-4101-b7d6-6f86e30aa78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.700186 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57f3ea-be78-4101-b7d6-6f86e30aa78f admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.700409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57f3ea-be78-4101-b7d6-6f86e30aa78f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.700702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57f3ea-be78-4101-b7d6-6f86e30aa78f admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.701434 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57f3ea-be78-4101-b7d6-6f86e30aa78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.703088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e57f3ea-be78-4101-b7d6-6f86e30aa78f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.714734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1384/2757] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:05.719010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.729925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc8c62-9e40-452c-9a20-ab6aca927a26 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.734602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.735508 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.735786 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.736060 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] PATH_INFO: `/v3/projects/535b29b2ed784959bb912c5bd424094c/users/99911af06e6349c58bf6586004d592b5/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.751524 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=535b29b2ed784959bb912c5bd424094c, user_id=99911af06e6349c58bf6586004d592b5, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.754067 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.769330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1374/2758] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:05 2022] PUT /identity/v3/projects/535b29b2ed784959bb912c5bd424094c/users/99911af06e6349c58bf6586004d592b5/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:05.781539 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5acb-093d-441b-8698-913c73148281 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.781832 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5acb-093d-441b-8698-913c73148281 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.782320 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d5acb-093d-441b-8698-913c73148281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.811452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c18d5acb-093d-441b-8698-913c73148281 None None] MFA Rules not processed for user `99911af06e6349c58bf6586004d592b5`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:05.812716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c18d5acb-093d-441b-8698-913c73148281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:05.861133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:05.885697 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1385/2759] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:05.886971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:05.929318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:05.951634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f6ac8-6941-41b1-9a62-322cc76daf34 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:05.956259 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:05.956919 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:05.957203 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:05.957482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:05.958220 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:05.960781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:05.987343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1375/2760] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:03:05.991691 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d5acb-093d-441b-8698-913c73148281 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.002787 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d5acb-093d-441b-8698-913c73148281 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.060712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.061386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.061672 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.061971 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.062654 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.064657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.094431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1386/2761] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:05 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:06.098287 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.108779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66f71a-544f-40fc-9820-cc762200e3d1 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.113505 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.114159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.114443 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.114711 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.115473 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.117386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.127839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1376/2762] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:06.132964 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.142822 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf160213-9c4e-462a-8c4d-c677eb3b6770 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.148709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86910c1-70b2-403a-8ecd-b08b4ff342e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.149366 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86910c1-70b2-403a-8ecd-b08b4ff342e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.149649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86910c1-70b2-403a-8ecd-b08b4ff342e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.149922 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86910c1-70b2-403a-8ecd-b08b4ff342e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.150644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86910c1-70b2-403a-8ecd-b08b4ff342e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.152269 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86910c1-70b2-403a-8ecd-b08b4ff342e7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.161999 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1387/2763] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:06.166115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.176147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced64d9a-31dd-42b4-894c-6ebb341751ac admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.181058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.181933 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.182213 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.182489 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] PATH_INFO: `/v3/projects/3e8cdd4a755544d9bfc972d9fac34923/users/a07381a0871445f4b3a460a0261ac9b2/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.197781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e8cdd4a755544d9bfc972d9fac34923, user_id=a07381a0871445f4b3a460a0261ac9b2, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.200520 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.214279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1377/2764] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:06 2022] PUT /identity/v3/projects/3e8cdd4a755544d9bfc972d9fac34923/users/a07381a0871445f4b3a460a0261ac9b2/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:06.224893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.225272 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.225639 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.254840 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] MFA Rules not processed for user `a07381a0871445f4b3a460a0261ac9b2`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:06.256023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:06.321351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1388/2765] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:06.325845 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.387324 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804deefc-73bb-459b-b2c4-bd52176fb1b5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.391951 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.392610 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.392896 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.393176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.393873 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.395808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.422362 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1378/2766] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:06 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:06.426407 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.437929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee58a46-0f40-4549-b329-4d417146c1a9 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.515032 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:06.522988 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.523959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.524527 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.524801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.525625 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.527590 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.549043 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1389/2767] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:06 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:06.553311 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.564839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17574d-277c-4d68-8ccf-6d9e0a6e1d18 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.570310 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.571124 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.571470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.571841 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.572692 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.574465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.590782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1379/2768] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:06.590923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.599209 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c92dc5-d4d4-4d98-8926-af98ed2ebc7e admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.605293 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.606177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.606465 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.606739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] PATH_INFO: `/v3/projects/07303586d3494f5ba6c331c4555f1e4d/users/dac432a230c5428fba13f87bc6417efe/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.622296 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07303586d3494f5ba6c331c4555f1e4d, user_id=dac432a230c5428fba13f87bc6417efe, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.625251 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.639936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1390/2769] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:06 2022] PUT /identity/v3/projects/07303586d3494f5ba6c331c4555f1e4d/users/dac432a230c5428fba13f87bc6417efe/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:06.643946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.655321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984878a-8d95-4533-90e9-511799c00f18 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.712525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017af221-6a9c-44b7-a28e-d7cb388f60e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.713195 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af221-6a9c-44b7-a28e-d7cb388f60e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.713479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af221-6a9c-44b7-a28e-d7cb388f60e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.713758 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017af221-6a9c-44b7-a28e-d7cb388f60e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.714483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017af221-6a9c-44b7-a28e-d7cb388f60e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.716119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017af221-6a9c-44b7-a28e-d7cb388f60e7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.718187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:06.727881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1380/2770] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:06.731897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.742989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb135b-5aaa-425d-82f2-600ee3ff974f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.747952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.748838 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.749121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.749398 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] PATH_INFO: `/v3/projects/07303586d3494f5ba6c331c4555f1e4d/users/dac432a230c5428fba13f87bc6417efe/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.751677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07303586d3494f5ba6c331c4555f1e4d, user_id=dac432a230c5428fba13f87bc6417efe, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:06.754315 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:06.764796 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1391/2771] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:06 2022] PUT /identity/v3/projects/07303586d3494f5ba6c331c4555f1e4d/users/dac432a230c5428fba13f87bc6417efe/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:06.774839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.775163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.775466 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:06.801563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] MFA Rules not processed for user `dac432a230c5428fba13f87bc6417efe`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:06.802770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:06.872900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1381/2772] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:06.884631 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:06.886347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:06.947392 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a79e6-3165-4ed9-9498-67e478618d1a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:06.951111 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:06.951764 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:06.952051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:06.952330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:07.011510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:07.012963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:07.021959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1392/2773] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:10.614432 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:10.616024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:10.628606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449ecfa-d9bd-4421-b468-794b2a1a38db None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:10.632892 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:10.633548 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:10.633825 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:10.634127 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:10.700293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:10.701646 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:10.711402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1382/2774] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:11.446350 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:11.459515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f06f0-e453-44d3-89c9-e87c3bda4cde service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:11.465611 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf572-75df-4bec-aa6c-4465a85b015f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:11.466197 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf572-75df-4bec-aa6c-4465a85b015f service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:11.466485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf572-75df-4bec-aa6c-4465a85b015f service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:11.466757 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf572-75df-4bec-aa6c-4465a85b015f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:11.479865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bf572-75df-4bec-aa6c-4465a85b015f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:11.481333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bf572-75df-4bec-aa6c-4465a85b015f service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:11.494630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1393/2775] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:12.717438 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:12.718939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:12.798797 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb83730-7de1-4459-b482-34e6a03e0f30 service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:12.803514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:12.804403 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:12.804684 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:12.804958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] PATH_INFO: `/v3/users/fd68b7e317d1417e81eefba759004d99` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:12.806227 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd68b7e317d1417e81eefba759004d99)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:12.808272 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:12.838506 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:12.869960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd68b7e317d1417e81eefba759004d99'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:12.885304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd68b7e317d1417e81eefba759004d99'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:12.893563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd68b7e317d1417e81eefba759004d99'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:12.903731 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd68b7e317d1417e81eefba759004d99'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:12.904830 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd68b7e317d1417e81eefba759004d99'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:12.915754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1383/2776] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:12 2022] DELETE /identity/v3/users/fd68b7e317d1417e81eefba759004d99 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:03:12.920070 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:12.920990 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:12.921473 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:12.945929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:12.945929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:12.999134 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:13.015450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1394/2777] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:03:13.026162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.029286 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:13.094010 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39582a7-1c5f-4da1-8512-d7a47d97252d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.097813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700d8c1-d9b1-4792-9cdb-8be1ee1c78b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.098465 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700d8c1-d9b1-4792-9cdb-8be1ee1c78b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.098747 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700d8c1-d9b1-4792-9cdb-8be1ee1c78b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.099032 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700d8c1-d9b1-4792-9cdb-8be1ee1c78b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.110002 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700d8c1-d9b1-4792-9cdb-8be1ee1c78b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.111483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700d8c1-d9b1-4792-9cdb-8be1ee1c78b6 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.122629 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1384/2778] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:13.439047 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.440799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:13.462902 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb5bf5-e315-4930-98ce-07f2cbb58417 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.468876 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.469633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.469918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.470196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] PATH_INFO: `/v3/projects/40f8d45a1208441eb16991176526d242` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.471459 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40f8d45a1208441eb16991176526d242)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.473391 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.526201 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40f8d45a1208441eb16991176526d242', 'request_id': 'req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:13.532690 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40f8d45a1208441eb16991176526d242', 'request_id': 'req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:13.540206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40f8d45a1208441eb16991176526d242', 'request_id': 'req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:13.554549 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] The token cache is being invalidate because project 40f8d45a1208441eb16991176526d242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:13.555114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:13.555946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-ad3f6f03-2829-4308-91b0-4b4b2049bd2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:13.557713 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1395/2779] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:13 2022] DELETE /identity/v3/projects/40f8d45a1208441eb16991176526d242 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:13.562378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.562697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.562971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.582053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:13.583581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:13.661305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1385/2780] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:13.671015 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.671365 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.671644 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.691224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:13.692441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:13.702164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1396/2781] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:13.706781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.720720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a79c2c-f1cf-420a-957c-0ac3a75c9b02 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.726003 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.726666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.726945 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.727256 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.727992 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:03:13.728298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.729897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.743230 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1386/2782] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:03:13.747698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.758427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c8356-bc46-4039-be67-f1e66e02050c None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.763665 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.764335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.764620 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.764959 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.765701 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.767805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.795497 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1397/2783] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:13 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:13.799756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.810973 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24fc87e-2593-412b-a358-490eee77cb56 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.843614 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:13.877260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.877950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.878342 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.878589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.879576 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.881276 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.902733 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1387/2784] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:13 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:13.906883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.918543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09571a39-aea6-4cb6-ab61-262e1cc5b461 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.923324 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.924003 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.924291 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.924571 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.925331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.926961 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.939485 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1398/2785] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:13.943530 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:13.954070 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca4409-5212-46ac-831b-0702174558f5 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:13.959332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2204367b-7fb2-4ed8-9a5f-5eec45404ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:13.960773 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204367b-7fb2-4ed8-9a5f-5eec45404ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:13.961086 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204367b-7fb2-4ed8-9a5f-5eec45404ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:13.961332 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2204367b-7fb2-4ed8-9a5f-5eec45404ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:13.962185 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204367b-7fb2-4ed8-9a5f-5eec45404ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:13.963940 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2204367b-7fb2-4ed8-9a5f-5eec45404ed8 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:13.984997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1388/2786] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:13.989936 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:14.001244 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39341647-1f37-470f-a968-fea3fcb80cec admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:14.015345 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:14.016276 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:14.016562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:14.016847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] PATH_INFO: `/v3/projects/a5bb3207d61c4a8a8dff33596b7da179/users/dd35da7ac4eb4b1895f5012ccb4f880d/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:14.036010 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb3207d61c4a8a8dff33596b7da179, user_id=dd35da7ac4eb4b1895f5012ccb4f880d, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:14.039486 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:14.054653 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1399/2787] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:13 2022] PUT /identity/v3/projects/a5bb3207d61c4a8a8dff33596b7da179/users/dd35da7ac4eb4b1895f5012ccb4f880d/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:14.070329 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:14.070634 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:14.070915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:14.108920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] MFA Rules not processed for user `dd35da7ac4eb4b1895f5012ccb4f880d`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:14.110171 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:14.188181 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1389/2788] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:14.224203 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:14.226680 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:14.291133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add856cc-10fc-49e2-bba0-b16f545a8a71 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:14.295167 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281029d6-7aac-4d73-afef-4634db2ae921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:14.295894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281029d6-7aac-4d73-afef-4634db2ae921 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:14.296183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281029d6-7aac-4d73-afef-4634db2ae921 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:14.296469 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281029d6-7aac-4d73-afef-4634db2ae921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:14.306154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281029d6-7aac-4d73-afef-4634db2ae921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:14.307925 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281029d6-7aac-4d73-afef-4634db2ae921 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:14.317819 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1400/2789] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:14.338150 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:14.416349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a3c88-df31-413e-b183-8a587e190f67 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:14.420881 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:14.421768 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:14.422052 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:14.422331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] PATH_INFO: `/v3/users/dd35da7ac4eb4b1895f5012ccb4f880d` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:14.423593 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd35da7ac4eb4b1895f5012ccb4f880d)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:14.425515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:14.468987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:14.474543 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd35da7ac4eb4b1895f5012ccb4f880d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:14.487961 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd35da7ac4eb4b1895f5012ccb4f880d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:14.495952 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd35da7ac4eb4b1895f5012ccb4f880d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:14.504654 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd35da7ac4eb4b1895f5012ccb4f880d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:14.505475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd35da7ac4eb4b1895f5012ccb4f880d'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:14.513862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1390/2790] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:14 2022] DELETE /identity/v3/users/dd35da7ac4eb4b1895f5012ccb4f880d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:14.518800 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:14.519113 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:14.519396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:14.536667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:14.537904 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:14.546715 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:14.611841 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1401/2791] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:14.623683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:14.625451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:14.687372 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165ca300-9e37-49a0-8f52-54c6bc3c6ad4 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:14.692280 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351cb12b-12ab-4086-a1cc-f9f9a6f589fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:14.692967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cb12b-12ab-4086-a1cc-f9f9a6f589fb service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:14.693228 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cb12b-12ab-4086-a1cc-f9f9a6f589fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:14.693500 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351cb12b-12ab-4086-a1cc-f9f9a6f589fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:14.704374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cb12b-12ab-4086-a1cc-f9f9a6f589fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:14.705891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351cb12b-12ab-4086-a1cc-f9f9a6f589fb service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:14.718065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1391/2792] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:15.024808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.026923 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.057893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db54f0-0875-4624-86d3-ea8ca49fa602 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.063967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.065831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.066129 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.066441 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] PATH_INFO: `/v3/projects/a5bb3207d61c4a8a8dff33596b7da179` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.069583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5bb3207d61c4a8a8dff33596b7da179)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.072512 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.138067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5bb3207d61c4a8a8dff33596b7da179', 'request_id': 'req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.145248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5bb3207d61c4a8a8dff33596b7da179', 'request_id': 'req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.153267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5bb3207d61c4a8a8dff33596b7da179', 'request_id': 'req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.176471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] The token cache is being invalidate because project a5bb3207d61c4a8a8dff33596b7da179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:15.176771 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.177443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-88f0aaa4-f9ba-4e09-9b79-e813ba15a8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.180143 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1402/2793] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:15 2022] DELETE /identity/v3/projects/a5bb3207d61c4a8a8dff33596b7da179 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:15.188749 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.189519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.189519 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.208491 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:15.210445 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.277218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1392/2794] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:15.285183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.285468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.285739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.305492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:15.306712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.315252 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1403/2795] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:15.320647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.331423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee15a5-2b49-41c0-a09e-ca01835ccef3 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.337366 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.338043 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.338326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.338601 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.339377 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:03:15.339686 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.341284 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.356145 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1393/2796] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:03:15.360717 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.371498 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41590f35-cfbf-4f10-92c9-69a3a0bf5371 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.376596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.377279 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.377563 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.377839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.378719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.380779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.416450 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1404/2797] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:15 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:15.422009 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.434219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea994df-9944-40f3-bdff-7433eb755093 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.466519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.468267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.484563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:15.502355 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.502908 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.503135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.503413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.504064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.505989 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.532499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1394/2798] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:15 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:15.537533 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.550607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca7d63-37c5-49c5-9fc4-87a9932a128d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.552208 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d212-e198-45de-9a55-77928cece710 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.558510 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.559479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.559756 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.560046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.560298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.560650 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.561333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.561333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] PATH_INFO: `/v3/users/99911af06e6349c58bf6586004d592b5` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.561537 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.562660 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=99911af06e6349c58bf6586004d592b5)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.563822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.565196 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.579667 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1395/2799] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:03:15.583594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.583867 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:15.598497 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedbc9c-84eb-4cc0-94dd-ee27873d3533 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.607830 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.608503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.608782 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.609058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.609784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.611589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.627567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1396/2800] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:03:15.632197 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.641999 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4221daa-2944-466b-ac16-20a8e68eb54a admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.644142 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99911af06e6349c58bf6586004d592b5'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.647559 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea22e7-1f34-4ba8-aa54-bb38ae4a7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.648517 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea22e7-1f34-4ba8-aa54-bb38ae4a7ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.648791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea22e7-1f34-4ba8-aa54-bb38ae4a7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.649064 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea22e7-1f34-4ba8-aa54-bb38ae4a7ff2 admin admin] PATH_INFO: `/v3/projects/5d906ca8441643369eacf6255bcacf3e/users/ae6e7a6e3cc949d9b83e4be12e4bab98/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.664020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99911af06e6349c58bf6586004d592b5'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.668481 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea22e7-1f34-4ba8-aa54-bb38ae4a7ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d906ca8441643369eacf6255bcacf3e, user_id=ae6e7a6e3cc949d9b83e4be12e4bab98, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.671176 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea22e7-1f34-4ba8-aa54-bb38ae4a7ff2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.675570 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99911af06e6349c58bf6586004d592b5'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.676361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99911af06e6349c58bf6586004d592b5'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.677018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-3f1acf28-27ec-4b23-a346-463f49bec7bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99911af06e6349c58bf6586004d592b5'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:15.688922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1397/2801] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:15 2022] PUT /identity/v3/projects/5d906ca8441643369eacf6255bcacf3e/users/ae6e7a6e3cc949d9b83e4be12e4bab98/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:03:15.692893 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1405/2802] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:15 2022] DELETE /identity/v3/users/99911af06e6349c58bf6586004d592b5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:03:15.696177 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.696503 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.696779 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.698568 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.698977 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.699383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.716826 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:15.718470 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.719687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:15.721085 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.791028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1398/2803] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:03:15.796376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1406/2804] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:03:15.805121 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.806819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.807502 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:15.808848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:15.867271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c76ff4-a98d-4a8f-a244-72a9adb29b9d None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.871304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.871977 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.872258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.872535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.874389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78802279-81c1-4d69-a38d-2a7794420847 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:15.878375 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:15.879037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:15.879355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:15.879630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:15.885216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.886656 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.893243 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:15.894687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:15.898207 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1399/2805] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:15.906635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1407/2806] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:03:16.248987 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:16.253741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:16.290795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328ac69-8d98-4e7d-8ff4-d9f6e324be79 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:16.298327 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:16.299485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:16.300061 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:16.300597 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] PATH_INFO: `/v3/projects/535b29b2ed784959bb912c5bd424094c` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:16.302449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=535b29b2ed784959bb912c5bd424094c)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:16.305360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:16.354097 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:16.384476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '535b29b2ed784959bb912c5bd424094c', 'request_id': 'req-35bb2306-b92c-48a4-8b60-6f85a606c7eb'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.392932 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '535b29b2ed784959bb912c5bd424094c', 'request_id': 'req-35bb2306-b92c-48a4-8b60-6f85a606c7eb'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.402260 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '535b29b2ed784959bb912c5bd424094c', 'request_id': 'req-35bb2306-b92c-48a4-8b60-6f85a606c7eb'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.422020 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] The token cache is being invalidate because project 535b29b2ed784959bb912c5bd424094c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:16.422330 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.423044 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-35bb2306-b92c-48a4-8b60-6f85a606c7eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.424772 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1400/2807] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:16 2022] DELETE /identity/v3/projects/535b29b2ed784959bb912c5bd424094c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:16.428952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:16.439337 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8570b214-3527-4bc1-8f00-c2e1cc283c10 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:16.474083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:16.509414 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:16.510460 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:16.510802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:16.511160 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] PATH_INFO: `/v3/users/a07381a0871445f4b3a460a0261ac9b2` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:16.512422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07381a0871445f4b3a460a0261ac9b2)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:16.514393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:16.568920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07381a0871445f4b3a460a0261ac9b2'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.588078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07381a0871445f4b3a460a0261ac9b2'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.596311 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07381a0871445f4b3a460a0261ac9b2'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.597347 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07381a0871445f4b3a460a0261ac9b2'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.598224 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07381a0871445f4b3a460a0261ac9b2'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:16.620630 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1408/2808] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:16 2022] DELETE /identity/v3/users/a07381a0871445f4b3a460a0261ac9b2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:16.714095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:16.714393 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:16.715132 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:16.750708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] MFA Rules not processed for user `ae6e7a6e3cc949d9b83e4be12e4bab98`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:16.752046 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:16.857544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1401/2809] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:16.869945 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:16.871597 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:16.957298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1d3e-66d1-45a2-b6c2-0beafebf0ea6 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:16.962702 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:16.963475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:16.963812 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:16.964089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:16.982059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:16.983607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:16.995906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:16.996976 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1409/2810] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:16.997486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:17.095247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2699d5-79e6-43ba-afab-5c995e99543e None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:17.100926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:17.102013 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:17.102297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:17.102572 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] PATH_INFO: `/v3/projects/3e8cdd4a755544d9bfc972d9fac34923` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:17.103925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e8cdd4a755544d9bfc972d9fac34923)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:17.105904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:17.170739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e8cdd4a755544d9bfc972d9fac34923', 'request_id': 'req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.172133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:17.182544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e8cdd4a755544d9bfc972d9fac34923', 'request_id': 'req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.192354 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e8cdd4a755544d9bfc972d9fac34923', 'request_id': 'req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.208828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] The token cache is being invalidate because project 3e8cdd4a755544d9bfc972d9fac34923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:17.209122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.210617 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.212422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1402/2811] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:16 2022] DELETE /identity/v3/projects/3e8cdd4a755544d9bfc972d9fac34923 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:17.217052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:17.229699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447326b5-e17b-4dec-9c8a-ec588908c0b4 service cinder] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:17.309115 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:17.328735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:17.331080 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:17.332024 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:17.334008 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] PATH_INFO: `/v3/users/dac432a230c5428fba13f87bc6417efe` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:17.336860 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac432a230c5428fba13f87bc6417efe)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:17.339436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:17.402056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac432a230c5428fba13f87bc6417efe'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.420103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac432a230c5428fba13f87bc6417efe'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.427117 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac432a230c5428fba13f87bc6417efe'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.427854 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac432a230c5428fba13f87bc6417efe'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.429681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-17e4f4d5-ab84-4a73-9c1f-2817ee59376b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac432a230c5428fba13f87bc6417efe'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.448906 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1410/2812] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:17 2022] DELETE /identity/v3/users/dac432a230c5428fba13f87bc6417efe => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:17.747719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:17.749417 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:17.764688 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb02a86-9d47-44c1-a1ce-a466c5b2327f admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:17.839631 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:17.840424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:17.840708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:17.840981 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] PATH_INFO: `/v3/projects/07303586d3494f5ba6c331c4555f1e4d` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:17.842198 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07303586d3494f5ba6c331c4555f1e4d)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:17.844161 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:17.898948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07303586d3494f5ba6c331c4555f1e4d', 'request_id': 'req-451d2daf-544c-4d6b-8284-bd8f0ca8d749'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.905565 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07303586d3494f5ba6c331c4555f1e4d', 'request_id': 'req-451d2daf-544c-4d6b-8284-bd8f0ca8d749'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.915238 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07303586d3494f5ba6c331c4555f1e4d', 'request_id': 'req-451d2daf-544c-4d6b-8284-bd8f0ca8d749'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.929725 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] The token cache is being invalidate because project 07303586d3494f5ba6c331c4555f1e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:17.930014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.930666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-451d2daf-544c-4d6b-8284-bd8f0ca8d749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:17.932241 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1403/2813] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:17 2022] DELETE /identity/v3/projects/07303586d3494f5ba6c331c4555f1e4d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:17.937667 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:17.937952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:17.938231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:17.956897 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:17.958171 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.025905 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1411/2814] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:18.033811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.034091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.034476 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.056815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:18.058042 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.066218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1404/2815] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:18.071258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.081205 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d36fba-2315-46be-a8b5-c3c2009f9f4d None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.087326 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.088072 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.088416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.088750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.091447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:03:18.091447 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.091718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.110044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1412/2816] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:03:18.117856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.127535 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c403d749-c9ee-43eb-9091-e4c7a520d1cd None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.138017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.138719 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.139006 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.139421 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] PATH_INFO: `/v3/projects` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.140451 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.142229 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.175982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1405/2817] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:18 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:18.180699 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.195014 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf4bd-4274-4c77-8b2a-83869dcb23bb admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.277827 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.278608 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.278890 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.279422 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] PATH_INFO: `/v3/users` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.280133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.282420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.290215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:18.311125 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.313334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.319374 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1413/2818] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:18 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:03:18.321325 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.334760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1813b118-cde1-4145-9743-38017a3463f0 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.340801 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.341235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.341602 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.341816 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.342565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.344222 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.355854 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1414/2819] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:18.360445 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.372543 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b25c91-15c1-4261-ba5b-742f3970d458 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.379042 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.380072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:18.380390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.381059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.381732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.383236 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.388743 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.401403 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1415/2820] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:03:18.405949 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0c1a9-b6ad-4fda-bb63-6123410c256e admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.406089 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.410472 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.411165 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.411453 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.411729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.417274 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.420802 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05bece-31a6-4668-a885-f1f626eadaa1 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.425947 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf2580-1e7f-4f37-ae3d-dad2ce339d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.427052 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf2580-1e7f-4f37-ae3d-dad2ce339d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.427479 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf2580-1e7f-4f37-ae3d-dad2ce339d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.427818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf2580-1e7f-4f37-ae3d-dad2ce339d9f admin admin] PATH_INFO: `/v3/projects/9cafc034cb864100bd74ee94b60cd686/users/4f938e167d2843bbb531ccf429ccdd34/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.446329 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf2580-1e7f-4f37-ae3d-dad2ce339d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cafc034cb864100bd74ee94b60cd686, user_id=4f938e167d2843bbb531ccf429ccdd34, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.449478 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf2580-1e7f-4f37-ae3d-dad2ce339d9f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.467480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1416/2821] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:18 2022] PUT /identity/v3/projects/9cafc034cb864100bd74ee94b60cd686/users/4f938e167d2843bbb531ccf429ccdd34/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:18.469283 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.469759 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.470036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.489059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] MFA Rules not processed for user `ae6e7a6e3cc949d9b83e4be12e4bab98`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:18.490298 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.504122 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.505635 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f683713-8e5b-4c7f-9bdc-35aff3d21507 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.522269 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1406/2822] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:03:18.528326 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.528590 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.528876 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.562760 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1417/2823] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:18.566783 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] MFA Rules not processed for user `4f938e167d2843bbb531ccf429ccdd34`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:18.568652 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.605300 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.605556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.658699 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1407/2824] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:18.670607 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.672147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.685921 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4207778-1f76-4b5d-9029-a0000ee99e78 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.689882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be5097840a23492585d297d35e2fea20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.690779 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.691083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.691361 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.693475 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:18.735379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9be71-274f-400f-96c9-786d960559d8 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.741956 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.742627 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.742918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.743233 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.752862 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.754317 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.762550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:18.764194 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1408/2825] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:03:18.781002 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.782687 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4dbb67-838f-4a77-b0a5-36a3ae4cd94e service glance] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.796965 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:18.797658 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1418/2826] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:03:18.812382 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=100 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:18.890563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a7739-1a7a-42e7-a8da-739e229a42ea service cinder] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:18.895267 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:18.896147 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:18.896423 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:18.896697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] PATH_INFO: `/v3/users/4f938e167d2843bbb531ccf429ccdd34` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:18.898659 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f938e167d2843bbb531ccf429ccdd34)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:18.901017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:18.959894 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f938e167d2843bbb531ccf429ccdd34'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:18.985958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f938e167d2843bbb531ccf429ccdd34'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:18.993534 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f938e167d2843bbb531ccf429ccdd34'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.002784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f938e167d2843bbb531ccf429ccdd34'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.003441 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f938e167d2843bbb531ccf429ccdd34'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.009869 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1409/2827] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:18 2022] DELETE /identity/v3/users/4f938e167d2843bbb531ccf429ccdd34 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:19.014645 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.014949 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.015439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.034381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:19.035624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:19.112162 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1419/2828] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:19.122004 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:19.123499 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:19.190028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0282a-5c07-46a0-9a75-bd4869c35919 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:19.194426 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ef8ea-cd3f-4a78-82a1-8783a10e4ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:19.194904 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ef8ea-cd3f-4a78-82a1-8783a10e4ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.195339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ef8ea-cd3f-4a78-82a1-8783a10e4ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.195630 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ef8ea-cd3f-4a78-82a1-8783a10e4ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.205897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ef8ea-cd3f-4a78-82a1-8783a10e4ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:19.207592 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ef8ea-cd3f-4a78-82a1-8783a10e4ca2 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:19.219468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1410/2829] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:19.613519 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:19.616120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:19.630848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4eca1-9d2c-426f-92f9-f09dc480af0b None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:19.640504 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:19.641420 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.641703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.641983 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] PATH_INFO: `/v3/projects/9cafc034cb864100bd74ee94b60cd686` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.643368 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cafc034cb864100bd74ee94b60cd686)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:19.645383 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:19.715732 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cafc034cb864100bd74ee94b60cd686', 'request_id': 'req-4bff9672-67c8-4508-9766-0949ec0199ba'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.723489 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cafc034cb864100bd74ee94b60cd686', 'request_id': 'req-4bff9672-67c8-4508-9766-0949ec0199ba'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.731888 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cafc034cb864100bd74ee94b60cd686', 'request_id': 'req-4bff9672-67c8-4508-9766-0949ec0199ba'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.750522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] The token cache is being invalidate because project 9cafc034cb864100bd74ee94b60cd686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:19.751169 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.752148 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-4bff9672-67c8-4508-9766-0949ec0199ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:19.753857 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1420/2830] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:19 2022] DELETE /identity/v3/projects/9cafc034cb864100bd74ee94b60cd686 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:19.759850 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.760399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.760399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.779557 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:19.780823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:19.821399 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:19.861704 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1411/2831] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:19.870476 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.870788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.871177 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.907950 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:19.909837 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:19.920646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1421/2832] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:19.925529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:19.936475 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb542f-2995-4549-a268-3c7f1122e1f4 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:19.941766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:19.941964 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.942139 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.942410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.943187 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:03:19.943568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:19.945708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:19.961159 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1412/2833] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:03:19.965448 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:19.975942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f2231-830a-4ac3-9439-f3c527b8c129 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:19.981105 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:19.981777 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:19.982059 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:19.982335 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:19.983034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:19.984960 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:20.015480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1422/2834] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:03:19 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:20.016918 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:20.028521 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773f221-f834-43de-bbaf-cf03ae4ca194 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:20.102105 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:20.102794 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:20.103101 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:20.103379 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:20.104192 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:20.106405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:20.130023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1413/2835] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:03:20 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:03:20.135027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:20.148012 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed023bf0-1a08-4feb-a650-897201734f7a admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:20.153788 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:20.154555 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:20.154766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:20.155040 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:20.155829 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:20.157452 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:20.170331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1423/2836] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:20.174305 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:20.185563 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7953ceb-5e7f-4c21-becc-d90d9ee28dda admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:20.190271 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bbe70-6fbb-48bb-8638-10441400c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:20.190946 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bbe70-6fbb-48bb-8638-10441400c962 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:20.191263 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bbe70-6fbb-48bb-8638-10441400c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:20.191542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bbe70-6fbb-48bb-8638-10441400c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:20.192283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bbe70-6fbb-48bb-8638-10441400c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:20.194568 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bbe70-6fbb-48bb-8638-10441400c962 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:20.205344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1414/2837] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:03:20.210396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:20.220938 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929a6cc-87b9-4432-8388-abbc277bba46 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:20.225685 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:20.226574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:20.226848 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:20.227140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] PATH_INFO: `/v3/projects/307559d7d9f040a59a101162b7a1a9ec/users/bda78d6ce6c04bfdb085715fe120b124/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:20.244987 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307559d7d9f040a59a101162b7a1a9ec, user_id=bda78d6ce6c04bfdb085715fe120b124, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:20.247942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:20.265062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1424/2838] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:03:20 2022] PUT /identity/v3/projects/307559d7d9f040a59a101162b7a1a9ec/users/bda78d6ce6c04bfdb085715fe120b124/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:20.275958 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:20.276249 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:20.276529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:20.305575 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] MFA Rules not processed for user `bda78d6ce6c04bfdb085715fe120b124`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:20.306855 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:20.376960 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1415/2839] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:20.388706 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:20.390428 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:20.402946 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:20.456537 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf4ed-3e47-49f5-aa71-ad7d05bbcbd2 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:20.464204 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8be4ef-13e7-4e36-94c0-2d6083a51bd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee043775ebb54bed8642c9f735b46c7f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:20.464891 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be4ef-13e7-4e36-94c0-2d6083a51bd4 service cinder] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:20.465178 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be4ef-13e7-4e36-94c0-2d6083a51bd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:20.465455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be4ef-13e7-4e36-94c0-2d6083a51bd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:20.476956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8be4ef-13e7-4e36-94c0-2d6083a51bd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:20.478663 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8be4ef-13e7-4e36-94c0-2d6083a51bd4 service cinder] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:20.493248 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1425/2840] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:30.373391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:30.484869 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=22 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:30.702980 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:30.771819 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efb35e-0cca-424d-b592-f7fb0c55a755 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:30.777990 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:30.778915 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:30.779297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:30.779579 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] PATH_INFO: `/v3/users/bda78d6ce6c04bfdb085715fe120b124` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:30.780795 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda78d6ce6c04bfdb085715fe120b124)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:30.782756 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:30.842709 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda78d6ce6c04bfdb085715fe120b124'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:30.856485 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda78d6ce6c04bfdb085715fe120b124'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:30.864304 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda78d6ce6c04bfdb085715fe120b124'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:30.873673 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda78d6ce6c04bfdb085715fe120b124'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:30.874468 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda78d6ce6c04bfdb085715fe120b124'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:30.882623 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1416/2841] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:30 2022] DELETE /identity/v3/users/bda78d6ce6c04bfdb085715fe120b124 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:30.887522 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:30.887814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:30.888256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:30.912767 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:30.914929 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:30.992894 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1426/2842] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:31.002988 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:31.004602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:31.073962 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707ed31-6e9d-454b-a294-a537854e1f62 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:31.077142 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:31.077926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:31.078283 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:31.078760 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:31.099976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:31.101741 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:31.117424 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1417/2843] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:31.408808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:31.411075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:31.428612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3338fc8f-6811-4de9-b7bf-993193466204 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:31.433545 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:31.434523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:31.434859 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:31.435379 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] PATH_INFO: `/v3/projects/307559d7d9f040a59a101162b7a1a9ec` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:31.437618 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] RBAC: Authorizing `identity:delete_project(project_id=307559d7d9f040a59a101162b7a1a9ec)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:31.440022 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:31.503302 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307559d7d9f040a59a101162b7a1a9ec', 'request_id': 'req-26a26d74-6015-43e2-a744-8ad066b2348f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:31.510805 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307559d7d9f040a59a101162b7a1a9ec', 'request_id': 'req-26a26d74-6015-43e2-a744-8ad066b2348f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:31.517351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307559d7d9f040a59a101162b7a1a9ec', 'request_id': 'req-26a26d74-6015-43e2-a744-8ad066b2348f'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:31.533803 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] The token cache is being invalidate because project 307559d7d9f040a59a101162b7a1a9ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:31.534091 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:31.535413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:31.537550 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1427/2844] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:31 2022] DELETE /identity/v3/projects/307559d7d9f040a59a101162b7a1a9ec => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:03:33.604886 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:33.613890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ffe65-263f-4f0c-8391-83ea468b0061 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:33.638360 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:33.675152 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:33.675803 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:33.676079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:33.676359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:33.678868 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:33.690705 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:33.692380 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:33.702750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1418/2845] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:38.530342 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:38.531926 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:38.545360 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a26d74-6015-43e2-a744-8ad066b2348f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:38.549009 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:38.549657 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:38.549939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:38.550215 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:38.553825 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:38.565358 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:38.566873 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:38.577372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1428/2846] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:38.662926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:38.671724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98838742-324f-460c-83aa-37cd0b3d1421 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:38.680345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:38.681041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:38.681339 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:38.681672 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:38.683589 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:38.763072 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:38.764581 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab64a160-f6a5-4e74-9da1-1960a2fdae5a service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:38.776806 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1419/2847] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:03:41.403963 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:41.405507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:41.481077 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839755cc-e732-4047-afff-6f16a6315266 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:41.485758 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:41.486632 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:41.486932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:41.487206 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] PATH_INFO: `/v3/users/6acd7109811a44b780cf61a5b707eebc` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:41.488431 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6acd7109811a44b780cf61a5b707eebc)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:41.491083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:41.505044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:41.554833 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6acd7109811a44b780cf61a5b707eebc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:41.570193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6acd7109811a44b780cf61a5b707eebc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:41.577132 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6acd7109811a44b780cf61a5b707eebc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:41.577879 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6acd7109811a44b780cf61a5b707eebc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:41.578503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6acd7109811a44b780cf61a5b707eebc'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:41.592724 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1429/2848] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:03:41 2022] DELETE /identity/v3/users/6acd7109811a44b780cf61a5b707eebc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:41.597301 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:41.597808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:41.597906 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:41.615901 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:03:41.617092 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:41.684461 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1420/2849] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:03:41.695499 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:41.696952 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:41.766872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce1f90-68be-444e-8f1e-0be9af81b87f admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:41.770627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:41.772018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:41.772305 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:41.772686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:41.785179 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:41.787705 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:41.798599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1430/2850] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:03:42.097146 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:03:42.100770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:03:42.114975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57218481-9961-4ce4-b477-9e82ac9d8c22 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:03:42.120410 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:03:42.121300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:03:42.121602 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:03:42.122290 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] PATH_INFO: `/v3/projects/f4a69d59d62145a5bda8136a0e3448c6` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:03:42.123182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a69d59d62145a5bda8136a0e3448c6)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:03:42.125369 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:03:42.186745 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:42.189754 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a69d59d62145a5bda8136a0e3448c6', 'request_id': 'req-d544efcc-2cd3-4526-8286-2fa28765cb7a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:42.199011 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a69d59d62145a5bda8136a0e3448c6', 'request_id': 'req-d544efcc-2cd3-4526-8286-2fa28765cb7a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:42.206866 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a69d59d62145a5bda8136a0e3448c6', 'request_id': 'req-d544efcc-2cd3-4526-8286-2fa28765cb7a'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:42.228486 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] The token cache is being invalidate because project f4a69d59d62145a5bda8136a0e3448c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:03:42.228839 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:42.229512 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-d544efcc-2cd3-4526-8286-2fa28765cb7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:03:42.231091 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1421/2851] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:03:42 2022] DELETE /identity/v3/projects/f4a69d59d62145a5bda8136a0e3448c6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:03:51.795038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:03:52.222813 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:04:24.071044 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:04:24.074154 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:24.163739 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31174d45-6cdf-4994-b750-2473386a38e4 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:04:24.238036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:04:24.238725 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:04:24.239226 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:04:24.239536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:04:24.241493 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:24.254310 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:04:24.256297 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:04:24.268016 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1431/2852] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:04:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:04:34.262887 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=62 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:04:35.398352 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:04:35.398920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:04:35.398920 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:04:35.416996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] MFA Rules not processed for user `ae6e7a6e3cc949d9b83e4be12e4bab98`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:04:35.418273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:35.479000 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1422/2853] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:04:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:04:35.726915 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:04:35.728436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:35.787331 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d76eb-b7e9-47c2-a24a-1bf6fd661544 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:04:35.791036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:04:35.791709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:04:35.791985 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:04:35.792258 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:04:35.794195 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:35.804808 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:04:35.806247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:04:35.814712 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1432/2854] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:04:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:04:36.145854 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:04:36.147386 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:36.224582 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197775f8-bee1-4511-bc6a-1fe2c604e3a9 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:04:36.230034 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:04:36.230697 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:04:36.230975 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:04:36.231291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:04:36.233095 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:36.245984 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:04:36.247642 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:04:36.258732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1423/2855] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:04:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:04:45.266246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:04:45.267818 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:45.288121 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecab7a-71f5-4c7f-a2fb-fd2eb13bdcf4 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:04:45.295084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:04:45.295751 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:04:45.296051 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:04:45.296303 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:04:45.298092 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:04:45.311621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:04:45.313246 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:04:45.330490 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1433/2856] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:04:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:04:46.253900 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=57 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:04:55.325390 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=32 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:27.864840 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:27.866291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:27.877926 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb6c41-f5fa-4335-86fc-59333a853bf3 service placement] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:27.881898 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:27.882544 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:27.882828 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:27.883133 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:27.884872 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:27.945727 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:27.947184 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:27.956251 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1424/2857] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:05:28.843163 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:28.844804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:28.868075 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc69cb-7801-4865-b6d2-a4abdf00bcc9 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:28.875273 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:28.875942 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:28.876214 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:28.876480 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:28.878418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:28.893189 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:28.896526 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:28.909164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1434/2858] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:05:32.876884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:32.886776 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432aee07-e301-4f4e-bbcf-c9d37c7672d6 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:32.890683 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:32.891374 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:32.891678 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:32.891950 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:32.894922 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:32.968411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:32.969885 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:32.980163 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1425/2859] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:05:37.221416 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:37.223397 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:37.242377 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f3753-57f6-46b1-8de9-5ceda9998659 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:37.247234 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:37.248127 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:37.248406 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:37.248681 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] PATH_INFO: `/v3/users/ae6e7a6e3cc949d9b83e4be12e4bab98` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:37.250253 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6e7a6e3cc949d9b83e4be12e4bab98)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:37.252221 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:37.309267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6e7a6e3cc949d9b83e4be12e4bab98'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.323769 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6e7a6e3cc949d9b83e4be12e4bab98'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.332140 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6e7a6e3cc949d9b83e4be12e4bab98'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.332932 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6e7a6e3cc949d9b83e4be12e4bab98'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.333565 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6e7a6e3cc949d9b83e4be12e4bab98'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.346780 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1435/2860] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:05:37 2022] DELETE /identity/v3/users/ae6e7a6e3cc949d9b83e4be12e4bab98 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:05:37.552884 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:37.569115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd0a2-9081-4f32-97a3-576e38f06576 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:37.670860 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:37.671796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:37.672135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:37.672414 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] PATH_INFO: `/v3/projects/5d906ca8441643369eacf6255bcacf3e` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:37.674026 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d906ca8441643369eacf6255bcacf3e)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:37.677027 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:37.729180 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:37.748202 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d906ca8441643369eacf6255bcacf3e', 'request_id': 'req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.758856 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d906ca8441643369eacf6255bcacf3e', 'request_id': 'req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.768216 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d906ca8441643369eacf6255bcacf3e', 'request_id': 'req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.786053 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] The token cache is being invalidate because project 5d906ca8441643369eacf6255bcacf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:05:37.787234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.788014 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:37.795172 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1426/2861] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:05:37 2022] DELETE /identity/v3/projects/5d906ca8441643369eacf6255bcacf3e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:05:39.890346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:39.891884 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:39.954137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:39.955633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:39.973495 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64b808-08c2-4029-a2b1-af2b08d03628 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:40.034247 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b562a01-ce79-4c1d-8ba5-925ea9b4fdce admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:40.039649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:40.040690 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:40.041090 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:40.042282 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] PATH_INFO: `/v3/users/24c9257cf9254b82a2b9fa5ee41243be` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:40.043293 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c9257cf9254b82a2b9fa5ee41243be)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:40.045359 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:40.061418 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:40.073642 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:40.075678 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:40.076914 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:40.077376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:40.080686 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:40.104709 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:40.107471 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:40.122439 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1436/2862] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:05:40.147017 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c9257cf9254b82a2b9fa5ee41243be'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.167079 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c9257cf9254b82a2b9fa5ee41243be'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.174732 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c9257cf9254b82a2b9fa5ee41243be'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.186594 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c9257cf9254b82a2b9fa5ee41243be'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.187408 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c9257cf9254b82a2b9fa5ee41243be'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.196529 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1427/2863] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:05:39 2022] DELETE /identity/v3/users/24c9257cf9254b82a2b9fa5ee41243be => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:05:40.401989 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:40.422599 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84869b1d-20fd-4c66-859a-c3f9986488fd service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:40.491231 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:40.492145 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:40.492421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:40.492693 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] PATH_INFO: `/v3/projects/2f83ca7e7fc844f180ae057a576f3723` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:40.493901 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f83ca7e7fc844f180ae057a576f3723)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:40.495851 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:40.542962 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f83ca7e7fc844f180ae057a576f3723', 'request_id': 'req-20afa144-bb38-482c-b431-4c7defc4acda'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.549067 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f83ca7e7fc844f180ae057a576f3723', 'request_id': 'req-20afa144-bb38-482c-b431-4c7defc4acda'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.555412 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f83ca7e7fc844f180ae057a576f3723', 'request_id': 'req-20afa144-bb38-482c-b431-4c7defc4acda'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.568104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] The token cache is being invalidate because project 2f83ca7e7fc844f180ae057a576f3723 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:05:40.568387 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.569048 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.570624 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1437/2864] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:05:40 2022] DELETE /identity/v3/projects/2f83ca7e7fc844f180ae057a576f3723 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:05:40.574823 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:40.576306 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:40.589861 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28298426-1916-4fa8-a6fb-25e2eb3d4dc9 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:40.631312 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:40.649564 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:40.650455 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:40.650738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:40.651008 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] PATH_INFO: `/v3/users/9a88a11f1ac54e568d40216f068ec612` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:40.652243 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a88a11f1ac54e568d40216f068ec612)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:40.654153 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:40.703633 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a88a11f1ac54e568d40216f068ec612'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.716344 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a88a11f1ac54e568d40216f068ec612'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.724291 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a88a11f1ac54e568d40216f068ec612'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.731822 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a88a11f1ac54e568d40216f068ec612'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.732515 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-f72e3617-f7e4-4acb-9995-0a79c60b4c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a88a11f1ac54e568d40216f068ec612'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:40.738766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1428/2865] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:05:40 2022] DELETE /identity/v3/users/9a88a11f1ac54e568d40216f068ec612 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:05:40.915714 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:40.917233 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:40.930605 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afa144-bb38-482c-b431-4c7defc4acda admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:40.991453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:40.992333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:40.992612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:40.992878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] PATH_INFO: `/v3/projects/36901fd028e04a469089b3c5daa49bfd` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:40.994088 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36901fd028e04a469089b3c5daa49bfd)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:40.996104 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.022235 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:41.045831 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36901fd028e04a469089b3c5daa49bfd', 'request_id': 'req-35df9c9b-5f26-4b45-a1d9-15f16833bf43'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:41.052277 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36901fd028e04a469089b3c5daa49bfd', 'request_id': 'req-35df9c9b-5f26-4b45-a1d9-15f16833bf43'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:41.057994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36901fd028e04a469089b3c5daa49bfd', 'request_id': 'req-35df9c9b-5f26-4b45-a1d9-15f16833bf43'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:41.073120 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] The token cache is being invalidate because project 36901fd028e04a469089b3c5daa49bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:05:41.073409 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:41.074038 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-35df9c9b-5f26-4b45-a1d9-15f16833bf43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:05:41.075734 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1438/2866] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:05:40 2022] DELETE /identity/v3/projects/36901fd028e04a469089b3c5daa49bfd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:05:41.081164 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.082250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.082250 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.100724 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:05:41.101883 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:41.163867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1429/2867] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:05:41.171584 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.172034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.172443 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.191716 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:05:41.193254 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:41.201746 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1439/2868] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:05:41.206106 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.216650 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1442033-5020-4e9b-bfa3-df326e0d3944 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.221261 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.221917 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.222224 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.222492 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] PATH_INFO: `/v3/domains` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.223297 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86636) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:05:41.223560 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.225148 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.237218 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1430/2869] 10.222.0.86 () {58 vars in 1152 bytes} [Wed Jul 20 22:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:05:41.241614 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.251939 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccab64-19c1-47f3-9720-bb03d51000a2 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.258103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.258789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.259118 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.259386 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.260119 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.262036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.290592 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1440/2870] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:05:41 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:05:41.297766 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.318100 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887b8f4-0e9a-4a86-b1a9-9c2ddcb3580d admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.380402 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:41.396205 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.396878 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.397162 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.397439 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.398166 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.400120 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.422698 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1431/2871] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:05:41 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:05:41.427886 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.438882 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c74003-697a-46ed-8571-246872893999 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.444589 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.445351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.445700 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.446156 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.446958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.448814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.460948 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1441/2872] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:05:41.464857 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.475223 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc192877-d65c-49f1-ae2a-e00af08aec22 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.480076 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0e739-0242-4b47-933d-1758d4820ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.480750 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0e739-0242-4b47-933d-1758d4820ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.481030 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0e739-0242-4b47-933d-1758d4820ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.481300 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0e739-0242-4b47-933d-1758d4820ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.482028 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0e739-0242-4b47-933d-1758d4820ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.483827 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0e739-0242-4b47-933d-1758d4820ba2 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.497051 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1432/2873] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:05:41.502468 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.513363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5b9c-1bd5-4feb-ae38-a1976238916c admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.518307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.519389 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] REQUEST_METHOD: `PUT` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.519668 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.519965 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] PATH_INFO: `/v3/projects/edb66957f3a041039ae0d6ac39476549/users/40553bbbf7af408dbffb8da1beab1062/roles/daae1e7e6e714a95b85fff4ea4b37745` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.537381 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb66957f3a041039ae0d6ac39476549, user_id=40553bbbf7af408dbffb8da1beab1062, role_id=daae1e7e6e714a95b85fff4ea4b37745)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.540219 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.554937 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1442/2874] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:05:41 2022] PUT /identity/v3/projects/edb66957f3a041039ae0d6ac39476549/users/40553bbbf7af408dbffb8da1beab1062/roles/daae1e7e6e714a95b85fff4ea4b37745 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:05:41.560278 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f012851-894c-40de-b700-10f6deef886f None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.560567 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f012851-894c-40de-b700-10f6deef886f None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.560843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f012851-894c-40de-b700-10f6deef886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.578738 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-1f012851-894c-40de-b700-10f6deef886f None None] MFA Rules not processed for user `ec67f4a1435249d49af5a7f7df4f7a39`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:05:41.580041 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-1f012851-894c-40de-b700-10f6deef886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:41.643321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1433/2875] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:05:41.655774 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:41.658413 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:41.718049 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9d2fd-1ded-4565-8d86-6cd7e15f440b admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:41.721871 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:41.722564 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:41.723001 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:41.723395 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:41.733731 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:41.735241 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:41.746270 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1443/2876] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:05:45.940808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:45.941378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:45.941378 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:45.975897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] MFA Rules not processed for user `40553bbbf7af408dbffb8da1beab1062`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:05:45.975897 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:46.073373 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1434/2877] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:05:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:05:46.078036 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:46.088523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfe2a4-395c-4124-b232-72cc7e21ee8b service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:46.099865 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:46.100556 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:46.100849 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:46.101100 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:46.101920 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:46.103883 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:46.136627 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1444/2878] 10.222.0.86 () {60 vars in 1152 bytes} [Wed Jul 20 22:05:46 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:05:46.140720 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:46.152135 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790377-f7c3-4ccd-a8d7-ac64fa19c41b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:46.212815 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:46.221357 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:46.222780 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:46.223103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:46.223391 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] PATH_INFO: `/v3/users` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:46.224116 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:46.226069 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:46.249482 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1435/2879] 10.222.0.86 () {60 vars in 1143 bytes} [Wed Jul 20 22:05:46 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:05:46.255492 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:46.266466 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f71c3b-6a75-42ee-bb69-7be6b04db7d1 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:46.272735 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:46.273432 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:46.273718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:46.274000 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:46.274750 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:46.276457 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:46.292540 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1445/2880] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:05:46.292891 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:46.305729 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7211f3-eadb-4bed-8125-6e5acc31dae7 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:46.310843 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:46.312058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:46.312331 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:46.312611 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] PATH_INFO: `/v3/projects/ec618d0e6b064099a8c0ed0a4e22f102/users/3fac05a20b394eb0a5c088bb15095ef3/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:46.329151 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec618d0e6b064099a8c0ed0a4e22f102, user_id=3fac05a20b394eb0a5c088bb15095ef3, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:46.332349 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:46.355542 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1436/2881] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:05:46 2022] PUT /identity/v3/projects/ec618d0e6b064099a8c0ed0a4e22f102/users/3fac05a20b394eb0a5c088bb15095ef3/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:05:46.359474 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:46.371510 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f260da-d54a-4e52-a2f6-0b8097e34aa3 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:46.387114 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:05:46.456018 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880b8a2-516b-4978-a246-8a13f83b7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:46.456862 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880b8a2-516b-4978-a246-8a13f83b7ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:46.457330 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880b8a2-516b-4978-a246-8a13f83b7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:46.457612 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880b8a2-516b-4978-a246-8a13f83b7ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:46.458351 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880b8a2-516b-4978-a246-8a13f83b7ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:46.460247 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880b8a2-516b-4978-a246-8a13f83b7ec7 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:46.470718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1446/2882] 10.222.0.86 () {58 vars in 1121 bytes} [Wed Jul 20 22:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:05:46.475528 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:46.486771 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed5340-e516-451c-a02a-bc586a15b527 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:46.492137 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:46.493018 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] REQUEST_METHOD: `PUT` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:46.493334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:46.493613 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] PATH_INFO: `/v3/projects/ec618d0e6b064099a8c0ed0a4e22f102/users/3fac05a20b394eb0a5c088bb15095ef3/roles/62c943ce95d54a1999a447dca6a1103b` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:46.495660 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec618d0e6b064099a8c0ed0a4e22f102, user_id=3fac05a20b394eb0a5c088bb15095ef3, role_id=62c943ce95d54a1999a447dca6a1103b)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:46.498318 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:46.509334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1437/2883] 10.222.0.86 () {60 vars in 1482 bytes} [Wed Jul 20 22:05:46 2022] PUT /identity/v3/projects/ec618d0e6b064099a8c0ed0a4e22f102/users/3fac05a20b394eb0a5c088bb15095ef3/roles/62c943ce95d54a1999a447dca6a1103b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:05:51.249583 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] REQUEST_METHOD: `POST` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:51.250133 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:51.250372 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:51.296703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.auth.core [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] MFA Rules not processed for user `3fac05a20b394eb0a5c088bb15095ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=86635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:05:51.298596 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:51.397112 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1447/2884] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:05:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:05:51.414298 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:51.416939 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:51.509888 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb82229-6ec3-46b2-a479-10cce5221130 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:51.515389 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a18f9-a3bf-45a9-a880-4747f189de25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:51.516058 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a18f9-a3bf-45a9-a880-4747f189de25 service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:51.516334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a18f9-a3bf-45a9-a880-4747f189de25 service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:51.516608 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a18f9-a3bf-45a9-a880-4747f189de25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:51.580514 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a18f9-a3bf-45a9-a880-4747f189de25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:51.582387 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a18f9-a3bf-45a9-a880-4747f189de25 service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:51.591449 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1438/2885] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:05:55.415683 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:05:55.417346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:55.495005 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c6ffb-92ff-4807-b568-5fd3d31ea218 None None] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:05:55.500710 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:05:55.501393 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:05:55.501677 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:05:55.501956 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:05:55.504034 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:05:55.517336 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:05:55.519035 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:05:55.530562 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1448/2886] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:05:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:06:01.587411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:06:05.526332 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=87 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:06:08.910103 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:06:08.911295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:06:08.911295 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:06:08.930024 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] MFA Rules not processed for user `40553bbbf7af408dbffb8da1beab1062`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:06:08.931411 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:06:08.938457 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1439/2887] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:06:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:06:18.923842 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=3 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:06:29.908679 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:06:29.966289 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de4d43-188b-479d-8ff1-f359f263d496 service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:06:29.969872 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:06:29.970515 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:06:29.970791 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:06:29.971083 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:06:29.981911 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:06:29.983408 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:06:29.994919 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1449/2888] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:06:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:06:39.990267 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=26 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:08:29.909976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:08:29.920119 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07237a5-0bc2-43f5-b74d-a52a936d6d40 None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:08:29.924761 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:08:29.925412 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:08:29.925693 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:08:29.925969 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:08:29.927816 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:08:29.944219 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:08:29.945936 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2ff7a-6e64-4031-82a4-c24eaebc5aa9 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:08:29.960065 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1440/2889] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:08:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:08:31.169370 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:08:31.178909 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c9790-2277-47fa-b464-4766eb332d7f service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:08:31.182728 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:08:31.183427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:08:31.183703 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:08:31.183980 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:08:31.185804 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:08:31.203582 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:08:31.205137 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:08:31.216850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1450/2890] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:08:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:08:33.531784 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427a193-cf97-45c2-a45a-eb893b8a9f80 None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:08:33.532376 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427a193-cf97-45c2-a45a-eb893b8a9f80 None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:08:33.532495 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427a193-cf97-45c2-a45a-eb893b8a9f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:08:33.558762 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-6427a193-cf97-45c2-a45a-eb893b8a9f80 None None] MFA Rules not processed for user `40553bbbf7af408dbffb8da1beab1062`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:08:33.559970 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-6427a193-cf97-45c2-a45a-eb893b8a9f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:08:33.567050 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1441/2891] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:08:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:08:36.078536 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:08:36.088616 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc8f14-458f-4af7-b5cb-586676c826a9 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:08:36.092363 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:08:36.093711 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:08:36.094023 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:08:36.094505 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:08:36.096698 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:08:36.107507 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:08:36.111766 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:08:36.120026 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1451/2892] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:08:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:08:43.553437 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=9 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:08:46.115820 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=12 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:10:27.394676 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] REQUEST_METHOD: `POST` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:10:27.395345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:10:27.395345 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:10:27.412836 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.auth.core [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] MFA Rules not processed for user `40553bbbf7af408dbffb8da1beab1062`. Rule list: `[]` (Enabled: `True`). {{(pid=86636) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:10:27.414055 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:10:27.421183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1442/2893] 10.222.0.86 () {58 vars in 957 bytes} [Wed Jul 20 22:10:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:10:27.758455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:10:27.768718 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68904a7c-a53e-4b9d-ad56-ccbbbe6649a2 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:10:27.773282 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:10:27.773978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:10:27.774256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:10:27.774534 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:10:27.776436 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:10:27.787736 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:10:27.789168 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:10:27.798039 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1452/2894] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:10:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:10:29.287404 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:10:29.300234 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367d720-5176-4fcc-afb4-1649d1d6b81b None None] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:10:29.305319 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:10:29.305967 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:10:29.306253 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:10:29.306531 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:10:29.308321 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:10:29.321023 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:10:29.322606 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:10:29.332889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1443/2895] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:10:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:10:35.011346 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:10:35.030578 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599ed85-ff6d-48f7-b5f7-e47a8560c512 service neutron] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:10:35.036814 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:10:35.037633 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:10:35.037986 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:10:35.038333 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:10:35.040355 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:10:35.069784 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:10:35.071719 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:10:35.098958 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1453/2896] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:10:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:10:39.328483 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=9 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:10:45.080183 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=16 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:10:45.118398 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:10:45.120007 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:10:45.196539 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc220c-abc8-4119-92a5-bb54956e78de service placement] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:10:45.202235 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:10:45.202890 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:10:45.203200 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:10:45.203479 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:10:45.205285 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:10:45.218191 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:10:45.219769 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:10:45.230343 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1444/2897] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:10:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:10:55.225791 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=31 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:10:59.424847 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:10:59.484993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc948959-8357-4efd-a5b8-4560baa929d7 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:10:59.490102 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:10:59.490747 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:10:59.491030 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:10:59.491340 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:10:59.500839 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:10:59.502423 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:10:59.513216 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1454/2898] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:10:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:11:09.508164 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=27 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:12:24.590997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:24.592666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:12:24.626182 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932d758-7b1f-419b-9011-307e5d5d190e service nova] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:24.629879 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:24.630525 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:24.630811 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:24.631115 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:24.640138 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:24.641577 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:24.651381 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1445/2899] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:12:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:12:25.476569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:25.500421 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59af6-8f1d-4744-93b3-5448ed27b81f service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:25.506850 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e06439f92e494fb4a68ffee7bdb0ffc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:25.507727 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:25.508078 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:25.508429 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:25.510384 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:12:25.539256 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:25.540978 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:25.553027 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1455/2900] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:12:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:12:34.647089 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=15 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:12:35.548557 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=15 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:12:37.998943 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:38.007997 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d8999-12de-47a5-b306-c412eb3ba8a7 service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:38.011753 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f38c9f8b5ea438dbae61a15c1e84e1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:38.012334 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:38.012616 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:38.012889 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:38.083494 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:38.084948 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:38.105796 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1446/2901] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:12:40.995037 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:41.009084 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54a2d7-f0e6-4701-89aa-5aaab7e13256 service nova] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:41.016973 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:41.017635 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:41.018172 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:41.018453 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:41.020417 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:12:41.048376 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:41.050020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:41.061738 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1456/2902] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:12:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:12:44.177063 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:44.189632 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa3179-b9e7-4e24-b8ba-5432adf4d5be service neutron] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:44.195986 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:44.196649 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] REQUEST_METHOD: `GET` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:44.196976 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:44.197258 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:44.199864 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:12:44.212554 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:44.214513 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:44.225770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1447/2903] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:12:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:12:44.242878 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:44.256103 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955a606-c447-49d6-b07f-8add97f1102c service placement] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:44.263918 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:44.266062 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:44.266373 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:44.267621 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] PATH_INFO: `/v3/users/40553bbbf7af408dbffb8da1beab1062` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:44.269328 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40553bbbf7af408dbffb8da1beab1062)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:44.271574 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:44.324503 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40553bbbf7af408dbffb8da1beab1062'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.338020 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40553bbbf7af408dbffb8da1beab1062'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.348528 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40553bbbf7af408dbffb8da1beab1062'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.349193 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40553bbbf7af408dbffb8da1beab1062'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.349789 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40553bbbf7af408dbffb8da1beab1062'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.362569 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1457/2904] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:12:44 2022] DELETE /identity/v3/users/40553bbbf7af408dbffb8da1beab1062 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:12:44.543174 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:44.554982 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec8afa-4389-4892-97f0-d57bd0f0660d service placement] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:44.613215 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:44.613996 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:44.614273 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:44.614546 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] PATH_INFO: `/v3/projects/edb66957f3a041039ae0d6ac39476549` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:44.615739 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb66957f3a041039ae0d6ac39476549)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:44.617647 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:44.665925 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb66957f3a041039ae0d6ac39476549', 'request_id': 'req-e5f84029-2d60-425a-adac-4284ac344675'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.672770 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb66957f3a041039ae0d6ac39476549', 'request_id': 'req-e5f84029-2d60-425a-adac-4284ac344675'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.678279 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb66957f3a041039ae0d6ac39476549', 'request_id': 'req-e5f84029-2d60-425a-adac-4284ac344675'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.680781 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:12:44.706128 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] The token cache is being invalidate because project edb66957f3a041039ae0d6ac39476549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:12:44.706413 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.707039 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.708599 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1448/2905] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:12:44 2022] DELETE /identity/v3/projects/edb66957f3a041039ae0d6ac39476549 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:12:44.712607 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:44.714427 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:12:44.727797 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbc34e-fcc9-4448-acef-4656c1522f74 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:44.784307 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:44.785175 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:44.785455 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:44.785730 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] PATH_INFO: `/v3/users/3fac05a20b394eb0a5c088bb15095ef3` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:44.786994 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fac05a20b394eb0a5c088bb15095ef3)` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:44.788982 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:44.840056 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fac05a20b394eb0a5c088bb15095ef3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.842166 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:12:44.854180 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fac05a20b394eb0a5c088bb15095ef3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.861058 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fac05a20b394eb0a5c088bb15095ef3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.861741 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fac05a20b394eb0a5c088bb15095ef3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.862343 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.notifications [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fac05a20b394eb0a5c088bb15095ef3'} {{(pid=86635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:44.875444 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1458/2906] 10.222.0.86 () {58 vars in 1223 bytes} [Wed Jul 20 22:12:44 2022] DELETE /identity/v3/users/3fac05a20b394eb0a5c088bb15095ef3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:12:45.051022 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Authenticating user token {{(pid=86636) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:12:45.052578 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.fernet_utils [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86636) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:12:45.068807 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f84029-2d60-425a-adac-4284ac344675 admin admin] Validating token access rules against request {{(pid=86636) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:12:45.128808 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec67f4a1435249d49af5a7f7df4f7a39', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'feb77f81bd3a4077b594b0faf70919df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86636) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:12:45.129584 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:12:45.129867 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:12:45.130422 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] PATH_INFO: `/v3/projects/ec618d0e6b064099a8c0ed0a4e22f102` {{(pid=86636) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:12:45.131666 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec618d0e6b064099a8c0ed0a4e22f102)` {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:12:45.133573 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] RBAC: Authorization granted {{(pid=86636) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:12:45.179778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec618d0e6b064099a8c0ed0a4e22f102', 'request_id': 'req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:45.187183 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec618d0e6b064099a8c0ed0a4e22f102', 'request_id': 'req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:45.192708 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec618d0e6b064099a8c0ed0a4e22f102', 'request_id': 'req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9'} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:45.205487 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] The token cache is being invalidate because project ec618d0e6b064099a8c0ed0a4e22f102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86636) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:12:45.205778 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:45.206400 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG keystone.notifications [None req-5c8c39df-f7a9-47a6-ae46-bb330ecad8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86636) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:12:45.207971 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: [pid: 86636|app: 0|req: 1449/2907] 10.222.0.86 () {58 vars in 1232 bytes} [Wed Jul 20 22:12:45 2022] DELETE /identity/v3/projects/ec618d0e6b064099a8c0ed0a4e22f102 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:12:54.871866 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=11,keystone:DELETE=3,keystone:INSERT=1 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:12:55.202405 dv-cinder-849690-5-iscsi devstack@keystone.service[86636]: DEBUG dbcounter [-] [86636] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=86636) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:16:07.362967 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Authenticating user token {{(pid=86635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:16:07.364396 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:16:07.439280 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65838b40-b64d-419e-9bc7-a6187a6b1491 admin admin] Validating token access rules against request {{(pid=86635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:16:07.497587 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06938dbfe3df4bc28f065c82315e4308', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd00662c0e4ca4709b42718633ad60b71', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:16:07.498240 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] REQUEST_METHOD: `GET` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:16:07.498523 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] SCRIPT_NAME: `/identity` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:16:07.498799 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:16:07.500708 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.fernet_utils [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:16:07.513993 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:16:07.515649 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6809b-99e4-4d32-b8a9-4eb52798fa8c service placement] RBAC: Authorization granted {{(pid=86635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:16:07.525646 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: [pid: 86635|app: 0|req: 1459/2908] 10.222.0.86 () {60 vars in 1365 bytes} [Wed Jul 20 22:16:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:16:17.521249 dv-cinder-849690-5-iscsi devstack@keystone.service[86635]: DEBUG dbcounter [-] [86635] Writing DB stats keystone:SELECT=58 {{(pid=86635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}